Execute spread testing
Metadata
Current evaluation
Spread testing delivered: TestingService and an opt-in 'test' command merged via PR #694 (commit 4910b2d8), building on #565 and #592. Maintainer mr-cal closed the issue citing that PR.
Resolution: closed resolved
Reason: PR #694 ("feat: add a testing service and command", commit 4910b2d8) was merged, adding craft_application/services/testing.py which runs spread against the project's spread.yaml. Maintainer mr-cal closed the issue with the comment "Completed via https://github.com/canonical/craft-application/pull/694".
Evaluated by: mr-cal-bot
No scores available.
Issue body
### What needs to get done
Add a service that takes the test specification file (craft-specific spread.yaml) and executes the necessary tests. It may be necessary to specify an artifact to test, depending on the application (e.g. to test a pre-packed snap).
### Why it needs to get done
We need an easy and standard way to test craft projects.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Spread testing delivered: TestingService and an opt-in 'test' command merged via PR #694 (commit 4910b2d8), building on #565 and #592. Maintainer mr-cal closed the issue citing that PR. | |
| stealth/space-bunny-alpha | — | closed resolved | Implemented spread testing via an opt-in test command in canonical/craft-application PR #694, merged into the project. | |
| qwen/qwen3.6-35b-a3b | — | — | Implemented a service to execute spread tests for craft projects. The feature was completed and merged via pull request #694 in the canonical/craft-application repository. | |
| qwen/qwen3.8-27b |
Impact:
20
Quick Win:
8
Staleness:
78
Complexity:
60
Confidence:
90
Support Request:
0
|
close stale | Task to add a service executing spread tests from a craft-specific spread.yaml with optional artifact. Implemented via TestingService and the 'test' command (PR #694); spread.yaml since deprecated for <app-name>-test.yaml. Stale since 2025, only a bot comment. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
90
Complexity:
65
Confidence:
90
Support Request:
10
|
needs triage | Feature request to add a service for executing spread testing on craft projects via a craft-specific spread.yaml. Unlabelled, no maintainer response, inactive for over a year. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
80
Complexity:
65
Confidence:
85
Support Request:
10
|
needs triage | Feature request to add a service for executing spread tests on craft projects. Unlabeled, no maintainer response, and inactive for over a year. Requires prioritization and triage to determine current relevance. |
Update history
| Date | Change |
|---|---|
| closed |
Related work
-
Likely Fixed By:
canonical/craft-application#694
(confidence 97%)
Merged PR adding the TestingService and opt-in 'test' command; commit 4910b2d8 is the resolving change cited in the closing comment.
-
Related To:
canonical/craft-application#565
(confidence 85%)
Earlier merged PR 'feat(testing): process simplified spread.yaml' providing spread.yaml expansion that #694 builds upon.
-
Related To:
canonical/craft-application#592
(confidence 85%)
Earlier merged PR 'feat(testing): execute spread on generated file' covering spread execution on the generated test file.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1962 Spread test the documentation examples | charmcraft | open | Documentation examples are now covered by dedicated Spread tests and CI through merged PR #2126; the still-open issue should be closed. | |
| #592 feat(testing): execute spread on generated file | craft-application | merged | Merged #592: the test command now resolves the backend, writes a temporary spread.yaml, and executes Spread against the generated file. | |
| #16 tests: add spread support | rockcraft | merged | Merged PR #16 added Spread support for CRAFT-865 via commit 3aeba2b3 (merge commit 5bf2680a). | |
| #665 Start using `spread` for integration tests | charmcraft | closed | Adopted spread for integration tests via merged PR canonical/charmcraft#862 (merge commit 7790c044, branch spread-bootstrap). Issue closed by maintainer sergiusens on 2023-03-16 noting the work was done in that PR. | |
| #4433 tests(spread): add remote-build spread test | snapcraft | merged | Merged #4433 (commit 4a0a91117), adding weekly spread coverage for new and legacy remote-build implementations and resolving #4398. | |
| #291 Refactor spread testing to use spread extensions | fetch-service | closed | Fixed by merged PR #299 (commit 44d9906), which refactored spread tests to generate spread.yaml and use the new spread extension framework. | |
| #1366 tests(spread): make release test less flaky | charmcraft | merged | Merged #1366 at commit 05b18a03, reducing flakiness in the Spread release test and resolving #1362. | |
| #762 tests(spread): add a test for the fetch-service | rockcraft | merged | Merged PR #762 in commit f4cff41c, adding 51 lines of spread-test coverage for the fetch service across two files. | |
| #2173 tests: create basic integration test spread infrastructure | snapcraft | merged | Merged PR #2173 in commit ad7260ced, adding Spread integration-test infrastructure and porting the Go plugin integration test; it resolved LP #1779419. | |
| #723 fix(testing): more intelligent test handling | craft-application | merged | Merged PR #723 via commit 6b319b77, running Spread tests as root and strictly validating only top-level spread.yaml keys. |