tests: spread tests fail intermittently
Metadata
Current evaluation
Rockcraft spread tests intermittently fail because the docker snap daemon never starts in test VMs, and the retry loop is insufficient. Feb 2026 triage confirmed the same pattern is still in spread.yaml; a fix (diagnostics on failure) is suggested but no PR proposed.
Suggested action: keep open
Reason: The issue is labelled Status: Triaged and a maintainer's 2026-02-26 comment explicitly confirms it is 'still present in-tree (not obsolete)', with the current spread.yaml still using the same 'retry -n 10 --wait 2 sh -c "docker run --rm hello-world"' pattern and no diagnostics on failure. A concrete remediation (capture snap services/logs/journalctl on retry exhaustion) is proposed but no PR exists, so it remains a valid, open CI-reliability bug.
Impact:
35
Quick Win:
21.0
Staleness:
50
Complexity:
40
Confidence:
85
Support Request:
5
Issue body
### Overview
The spread tests are failing intermittently due to a docker error, `docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?`.
### Details
@tigarmo added the [`retry`](https://github.com/canonical/rockcraft/pull/94/files#diff-e1e5ecb5e7fe7aa228c64f680564673541f91452212c3c6c23cbbd27bb78dcedR47) command from [`snapd-testing-tools`](https://github.com/snapcore/snapd-testing-tools/blob/main/tools/retry), but we're still seeing the failure.
We have two more ideas:
1. Increase the `retry` timeout
2. [Run `snap logs docker`](https://github.com/canonical/rockcraft/pull/96/files) if docker never starts
### Logs
[source](https://github.com/canonical/rockcraft/actions/runs/3152544084/jobs/5128003514)
```
2022-09-29T15:30:03.8353470Z 2022-09-29 15:30:03 Preparing google:ubuntu-20.04-64 (sep291529-710396)...
2022-09-29T15:30:50.7936227Z 2022-09-29 15:30:50 Error preparing google:ubuntu-18.04-64 (sep291529-710434) :
2022-09-29T15:30:50.7937212Z -----
...
2022-09-29T15:30:50.8202707Z + snap install docker
2022-09-29T15:30:50.8203825Z docker 20.10.14 from Canonical** installed
2022-09-29T15:30:50.8204725Z + retry -n 10 --wait 2 sh -c 'docker run --rm hello-world'
2022-09-29T15:30:50.8205417Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8206155Z See 'docker run --help'.
2022-09-29T15:30:50.8206755Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8207558Z See 'docker run --help'.
2022-09-29T15:30:50.8208252Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8208953Z See 'docker run --help'.
2022-09-29T15:30:50.8209589Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8210394Z See 'docker run --help'.
2022-09-29T15:30:50.8211104Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8211805Z See 'docker run --help'.
2022-09-29T15:30:50.8212509Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8213310Z See 'docker run --help'.
2022-09-29T15:30:50.8213902Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8214654Z See 'docker run --help'.
2022-09-29T15:30:50.8215310Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8215947Z See 'docker run --help'.
2022-09-29T15:30:50.8216654Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8217364Z See 'docker run --help'.
2022-09-29T15:30:50.8217867Z docker: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?.
2022-09-29T15:30:50.8218561Z See 'docker run --help'.
2022-09-29T15:30:50.8219376Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8220001Z retry: next attempt in 2.0 second(s) (attempt 1 of 10)
2022-09-29T15:30:50.8220713Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8221349Z retry: next attempt in 2.0 second(s) (attempt 2 of 10)
2022-09-29T15:30:50.8222215Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8222833Z retry: next attempt in 2.0 second(s) (attempt 3 of 10)
2022-09-29T15:30:50.8223556Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8224459Z retry: next attempt in 2.0 second(s) (attempt 4 of 10)
2022-09-29T15:30:50.8225376Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8225965Z retry: next attempt in 2.0 second(s) (attempt 5 of 10)
2022-09-29T15:30:50.8227152Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8227788Z retry: next attempt in 2.0 second(s) (attempt 6 of 10)
2022-09-29T15:30:50.8228467Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8228917Z retry: next attempt in 2.0 second(s) (attempt 7 of 10)
2022-09-29T15:30:50.8229547Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8230011Z retry: next attempt in 2.0 second(s) (attempt 8 of 10)
2022-09-29T15:30:50.8230616Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8231159Z retry: next attempt in 2.0 second(s) (attempt 9 of 10)
2022-09-29T15:30:50.8231896Z retry: command sh -c docker run --rm hello-world failed with code 125
2022-09-29T15:30:50.8232682Z retry: command sh -c docker run --rm hello-world keeps failing after 10 attempts
2022-09-29T15:30:50.8233455Z -----
2022-09-29T15:30:50.8233874Z .
2022-09-29T15:30:50.8234466Z 2022-09-29 15:30:50 Discarding google:ubuntu-18.04-64 (sep291529-710434)...
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.8-27b |
Impact:
35
Quick Win:
21.0
Staleness:
50
Complexity:
40
Confidence:
85
Support Request:
5
|
keep open | Rockcraft spread tests intermittently fail because the docker snap daemon never starts in test VMs, and the retry loop is insufficient. Feb 2026 triage confirmed the same pattern is still in spread.yaml; a fix (diagnostics on failure) is suggested but no PR proposed. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
55
Complexity:
30
Confidence:
85
Support Request:
10
|
keep open | Intermittent spread test failures due to Docker daemon startup timeout. Triaged bug with recent maintainer suggestions for improved diagnostics, awaiting implementation. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
55
Complexity:
40
Confidence:
75
Support Request:
10
|
keep open | Intermittent spread test failures due to Docker daemon not starting during CI setup. Triaged and open, with recent suggestions to add diagnostic logging and increase retry timeouts. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
55
Complexity:
25
Confidence:
85
Support Request:
0
|
keep open | Intermittent spread test failures due to Docker daemon connection errors. Triaged and still present. Recent discussion recommends capturing diagnostic logs on failure and increasing retry duration to aid debugging. Awaiting a proposed fix. |
Update history
No update history recorded yet.
Related work
-
Related To:
charmcraft#1218
(confidence 35%)
Similar flaky spread test in the sibling project (store resources test), closed without a linked fix; useful precedent for how such CI flakes were handled.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #943 test(spread): test on more bases | craft-application | merged | Merged after passing CI checks and approvals. Updates spread tests to run across multiple Ubuntu bases using an external Python runtime snap, enabling early detection of base-specific bugs and serving as a prerequisite for ROCKCRAFT-255. | |
| #1304 test(spread): fix cross-test contamination | rockcraft | merged | Merged fix for spread test cross-contamination. Resolved by copying a clean rockcraft.yaml template at test start to isolate variants. Approved by two reviewers and passed CI checks. | |
| #665 create new Spread task for daily compatibility matrix testing | rockcraft | open | Feature request to add a dedicated daily spread test that runs rockcraft pack across multiple LXD/multipass/docker versions to catch dependency breakage early. Maintainer discussion on 2024-08-08; no implementation found in current codebase. | |
| #100 tests: print rockcraft log files on spread failure | rockcraft | merged | Merged: adds 11 lines to print rockcraft log files during spread test failures, improving debugging visibility. Approved by one reviewer and passed CI checks. |