ci: speed up spread test suite
Metadata
Current evaluation
Merged after approval, cutting CI spread test execution from 1h40m to 40m. Changes route slow tests to a dedicated workflow, use build wheels, add store retries, and fix the matter plugin test. Proceeded despite unrelated CI timeouts.
Suggested action: —
No scores available.
Issue body
- [x] Have you followed the [guidelines for contributing](https://github.com/canonical/snapcraft/blob/main/CONTRIBUTING.md)?
- [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [x] Have you successfully run `make lint`?
- [x] Have you successfully run `make test`?
---
This PR makes many improvements to the spread test suite to speed up the CI workflow.
Previously, the spread tests took 1 hour and 40 minutes to complete and 3 hours of total compute time.
Now, it takes 40 minutes to complete and 1 hour and 40 minutes of total compute time.
* Move extension, colcon, and core20 plugin tests to the slow workflow
* Allow triggering the slow workflow with a label ([using rockcraft's design](https://github.com/canonical/rockcraft/blob/main/.github/workflows/spread-manual.yaml))
* Use wheels when building the snap for spread tests
* Retry calls to the store that often fail due to store-side issues
* Fix the matter plugin spread test
Fixes #5296
(SNAPCRAFT-353)
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Merged after approval, cutting CI spread test execution from 1h40m to 40m. Changes route slow tests to a dedicated workflow, use build wheels, add store retries, and fix the matter plugin test. Proceeded despite unrelated CI timeouts. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged. Reduced spread test CI runtime from 1h40m to 40m by routing tests to a slow workflow, adding manual triggers, using build wheels, adding store retries, and fixing the matter plugin test. Unrelated CI failures were confirmed as external. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged. Optimized CI spread tests, cutting runtime from 1h40m to 40m. Changes include moving slow tests to a separate workflow, adding a manual trigger label, using build wheels, adding retries, and fixing the matter plugin test. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #432 ci: speed up spread testing workflow | fetch-service | merged | Merged CI workflow optimization that decouples snap-test from snap-build, reducing pipeline latency. The build job remains for artifact retrieval. Approved by two reviewers, passed all checks, and modified one file. | |
| #5954 tests: fix store tests | snapcraft | merged | Merged after approval. Updated store tests with new staging credentials and switched to wheels for automated spread tests. CI passed, with unrelated spread failures noted by the author. | |
| #3090 ci: move staging store tests to spread | snapcraft | merged | Merged into master. The change migrated staging store tests to the spread testing framework, removing 1434 lines across 27 files. Approved by two reviewers, passed CI, and achieved 100% diff coverage prior to integration. | |
| #1705 test: extend timeout for some spread tests | charmcraft | merged | Merged changes extending timeouts for specific spread tests to prevent failures from lengthy preparation steps. Approved by reviewers, passed CI, and modified two files. | |
| #1220 ci: always run spread workflow on schedule | rockcraft | merged | Merged CI configuration updates ensuring the full Spread test suite always runs on the scheduled bi-daily workflow for the main branch, preventing automatic skips. Approved by one reviewer and passed all checks. | |
| #280 ci, spread: add "large spreads" suite | rockcraft | merged | Merged addition of a new large spreads test suite to CI. Configured to run weekly via cron or manually via label. Approved by two reviewers and successfully integrated. | |
| #2126 ci(docs): spread test documentation | charmcraft | merged | Merged changes adding a CI workflow for spread testing documentation and adjusting tasks for Google runners. Approved by two reviewers, the update modifies five files (+100/-22 lines) and resolves CRAFT-3798. | |
| #3205 spread tests: higher timeout for extension tests | snapcraft | merged | Merged a pull request increasing the timeout for extension tests in the spread test suite. The single-line configuration change was approved by a reviewer, passed CI checks, and successfully integrated into the codebase. | |
| #2199 tests: add spread suite for copy plugin | snapcraft | merged | Merged into master following approval and passing CI. Moves copy plugin integration tests to a spread suite, resolving bug LP: #1783831. Modified 10 files, reduced code by 41 lines, and increased coverage by 0.02%. |