fix(testing): error early if spread.yaml is invalid.
Metadata
Current evaluation
Merged after two approvals. Adds early validation to fail tests immediately if spread.yaml is invalid. CI passed except for an unrelated snap-test failure, which was overridden.
Suggested action: —
No scores available.
Issue body
- [x] Have you followed the guidelines for contributing?
- [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [x] Have you successfully run `make lint && make test`?
- [x] Have you added an entry to the changelog (`docs/reference/changelog.rst`)?
---
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Merged after two approvals. Adds early validation to fail tests immediately if spread.yaml is invalid. CI passed except for an unrelated snap-test failure, which was overridden. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged a fix that fails the testing suite immediately upon detecting an invalid spread.yaml file. This improves early error detection during test execution. Unrelated fetch service failures were overridden during review. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged a fix that fails early when spread.yaml is invalid. The maintainer passed all lint and test checks, updated the changelog, and noted an override for unrelated fetch service failures during CI execution. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #723 fix(testing): more intelligent test handling | craft-application | merged | Merged after approval. Updates test handling to run spread tests as root and strictly parse top-level spread.yaml keys. Approved by two reviewers and passed CI checks before integration. | |
| #565 feat(testing): process simplified spread.yaml | craft-application | merged | Merged to resolve issue #560. Processes a user-supplied simplified spread.yaml into a complete project file for the spread testing framework. Approved by two reviewers, passed CI, and modified 11 files. | |
| #926 test: address a couple of non-flaky spread failures | rockcraft | merged | Merged after two approvals and passing all CI checks. Resolves non-flaky spread test failures by updating test configurations across three files, eliminating consistent failures. | |
| #534 test(spread): workaround for flaky test | fetch-service | merged | Merged a workaround for a flaky spread test (issue #533). The change adjusts one file (+2/-1 lines) to skip a no-op name query check while retaining other validations. Approved by two reviewers and passed all CI checks. | |
| #1084 tests: check for submodules before running spread | charmcraft | merged | Merged changes verifying the snapd-testing-tools submodule before running Spread tests. Adds a clear error if missing. Approved by a reviewer, passed CI, and merged with minimal modifications. | |
| #240 tests: check for submodules before running spread | rockcraft | merged | Merged. Adds a pre-check for the snapd-testing-tools submodule before running spread tests, providing a clear error if missing. Approved by two reviewers, passed CI, and merged successfully. | |
| #712 fix(TestingService): provide a better error if spread.yaml is missing | craft-application | merged | Merged update that improves error messaging when spread.yaml is missing in the TestingService. Approved by two reviewers, passed CI checks, and includes a changelog entry. Resolves #703. | |
| #5389 test(spread): trace logs in spread tests | snapcraft | merged | Merged after approval. Added trace logging to spread tests. Despite CI failures from extra snapcraft output, the change was approved and merged following a related feature branch integration to prevent conflicts. | |
| #5075 tests: fix validation sets spread test | snapcraft | merged | Merged fix for validation sets spread test switching to the hello-world snap and flipping the presence check instead of a specific revision. Approved by two reviewers with passing CI, resolving issue #5053. | |
| #62 spread: add a "big" spread test | rockcraft | merged | Merged a reusable spread test framework to house multiple checks without repeated setup. The initial test validates bug #44 regarding directory permissions. Approved by one reviewer and passed CI checks. |