Flaky spread tests
Metadata
Current evaluation
Closed without resolution. The flaky spread tests causing intermittent CI timeouts were never addressed, indicating the report was abandoned.
Suggested action: —
No scores available.
Issue body
### Bug Description
Some of our spread tests are flaky. Example failure: https://github.com/snapcore/snapcraft/actions/runs/4961379589/jobs/8878193699?pr=4128
They seem to be sometimes timing out, but not reliably so
### To Reproduce
1. Create a PR with any change.
2. Run the spread tests
3. If they all succeed, try re-running
### Environment
CI
### snapcraft.yaml
```shell
N/A
```
### Relevant log output
```shell
Loading plugin module 'nil' with sys.path ['/root/project/snap/plugins', '/snap/snapcraft/x1/bin', '/snap/snapcraft/x1/usr/lib/python38.zip', '/snap/snapcraft/x1/usr/lib/python3.8', '/snap/snapcraft/x1/usr/lib/python3.8/lib-dynload', '/snap/snapcraft/x1/lib/python3.8/site-packages']
Setting up part 'base' with plugin 'nil' and properties {'plugin': 'nil', 'stage': [], 'prime': []}.
Requested build-packages: []
Requested build-packages already installed: []
snapcraft is running in a docker or podman (OCI) container
Preparing to pull base
Pulling base
snapcraft is running as a snap True, SNAP_NAME set to 'snapcraft'
+ snapcraftctl pull
2023-05-12 17:24:11.077 Starting Snapcraft 7.3.1.post80+gitbe1f2db6
2023-05-12 17:24:11.077 Logging execution to '/root/.local/state/snapcraft/log/snapcraft-20230512-172411.075965.log'
2023-05-12 17:24:11.079 lifecycle command: 'pull', arguments: Namespace(bind_ssh=False, build_for=None, debug=False, destructive_mode=False, enable_developer_debug=False, enable_experimental_extensions=False, enable_experimental_plugins=False, enable_experimental_target_arch=False, enable_experimental_ua_services=False, enable_manifest=True, http_proxy=None, https_proxy=None, manifest_image_information=None, parts=[], provider=None, shell=False, shell_after=False, target_arch=None, ua_token=None, use_lxd=True)
2023-05-12 17:24:11.079 command: pull, arguments: Namespace(bind_ssh=False, build_for=None, debug=False, destructive_mode=False, enable_developer_debug=False, enable_experimental_extensions=False, enable_experimental_plugins=False, enable_experimental_target_arch=False, enable_experimental_ua_services=False, enable_manifest=True, http_proxy=None, https_proxy=None, manifest_image_information=None, parts=[], provider=None, shell=False, shell_after=False, target_arch=None, ua_token=None, use_lxd=True)
+ /snap/bin/lxc start snapcraft-core18
+ MATCH VERSION_CODENAME=bionic
+ /snap/bin/lxc exec snapcraft-core18 cat /etc/os-release
+ /snap/bin/lxc stop snapcraft-core18
Error: Failed shutting down instance, status is "Running": context deadline exceeded
Try `lxc info --show-log snapcraft-core18` for more info
-----
.
2023-05-12 17:36:43 Restoring google:ubuntu-18.04-64:tests/spread/providers/lxd-type-base (may121721-738851)...
```
### Additional context
_No response_
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Closed without resolution. The flaky spread tests causing intermittent CI timeouts were never addressed, indicating the report was abandoned. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without resolution. The flaky spread tests and LXD container timeout issues were not addressed, indicating the report was abandoned. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without resolution. The flaky spread tests causing LXC container shutdown timeouts were abandoned due to inactivity, with no fix implemented. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without resolution. Flaky spread tests timed out due to LXD container shutdown delays in CI. Zero comments and rapid closure indicate the issue was abandoned or resolved externally. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without resolution. The flaky spread test timeouts were abandoned due to lack of discussion and reproducible steps. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #533 Snap inspector spread test is flaky | fetch-service | closed | Closed without a documented fix. The flaky snap inspector spread test was reported but no resolution or merge was implemented. The issue appears abandoned or tracked internally via Jira. | |
| #1712 Store resources spread test is broken | charmcraft | closed | Closed without a fix or resolution. The CI test failure for resources spread was reported but zero comments or linked PRs indicate it was abandoned or marked stale. |