fix(testing): error early if spread.yaml is invalid.
Metadata
Current evaluation
PR #1029 merged (commit 93c27ec5): TestingService now reads and validates spread.yaml before starting the lifecycle, erroring early on invalid files. Changelog entry added under 6.3.0.
Resolution: closed resolved
Reason: The PR was approved and merged into main as commit 93c27ec5, touching5 files (+73/-15), and docs/reference/changelog.rst for 6.3.0 records 'The ``test`` command now reads ``spread.yaml`` before beginning the lifecycle, erroring early if the file is invalid.'
Evaluated by: mr-cal-bot
Issue body
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | PR #1029 merged (commit 93c27ec5): TestingService now reads and validates spread.yaml before starting the lifecycle, erroring early on invalid files. Changelog entry added under 6.3.0. | |
| stealth/space-bunny-alpha | — | closed resolved | Merged validation changes to fail early when spread.yaml is invalid, with +73/−15 lines across five files. | |
| qwen/qwen3.6-35b-a3b | — | — | Merged after two approvals. Adds early validation to fail tests immediately if spread.yaml is invalid. CI passed except for an unrelated snap-test failure, which was overridden. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged a fix that fails the testing suite immediately upon detecting an invalid spread.yaml file. This improves early error detection during test execution. Unrelated fetch service failures were overridden during review. |
Update history
No update history recorded yet.
Related work
-
Related To:
canonical/craft-application#1168
(confidence 60%)
Later PR 'feat(test): parse <app-name>-test.yaml, deprecate spread.yaml' (868462ad) supersedes the spread.yaml format this PR validates.
-
Likely Fixed By:
canonical/craft-application#1013
(confidence 55%)
Preceding commit f4a4d282 'fix(LifecycleService): pull local test files' that made the test files available early enough for this validation.
-
Related To:
canonical/craft-application#1036
(confidence 45%)
CI job disabling the external fetch service spread test, matching the PR comment 'Overriding unrelated fetch service failures' and the failing snap-tests (google:) check.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #703 Bad error message if 'spread.yaml' is missing | craft-application | closed | Fixed by merged PR #712 (commit f72dc340), which improved TestingService's error when spread.yaml is missing. | |
| #712 fix(TestingService): provide a better error if spread.yaml is missing | craft-application | merged | Merged as commit f72dc340, PR #712 fixed #703 by giving TestingService a clearer error when spread.yaml is missing. | |
| #723 fix(testing): more intelligent test handling | craft-application | merged | Merged PR #723 via commit 6b319b77, running Spread tests as root and strictly validating only top-level spread.yaml keys. | |
| #701 fix(TestingService): error better when we can't find spread | craft-application | merged | Merged PR #701 as bb2bda9c, improving the TestingService error when spread is missing from PATH; fixes #697. | |
| #565 feat(testing): process simplified spread.yaml | craft-application | merged | Merged PR #565 added simplified spread.yaml expansion into a complete project file, resolving #560 after approval by two reviewers. | |
| #613 Execute spread testing | craft-application | closed | Spread testing delivered: TestingService and an opt-in 'test' command merged via PR #694 (commit 4910b2d8), building on #565 and #592. Maintainer mr-cal closed the issue citing that PR. | |
| #291 Refactor spread testing to use spread extensions | fetch-service | closed | Fixed by merged PR #299 (commit 44d9906), which refactored spread tests to generate spread.yaml and use the new spread extension framework. |