test: use MATCH in spread tests
Metadata
Current evaluation
Merged in charmcraft#1382 (commit 90f23622), replacing grep assertions with MATCH in four spread-test files so missing expected output correctly fails tests.
Resolution: closed resolved
Reason: PR charmcraft#1382 was approved and merged as commit 90f23622. It hardened spread-test assertions by using MATCH so unexpected or missing output produces a failure.
Evaluated by: mr-cal-bot
No scores available.
Issue body
We've got a few places where we use grep directly in a way that doesn't guarantee test failure when it should.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Merged in charmcraft#1382 (commit 90f23622), replacing grep assertions with MATCH in four spread-test files so missing expected output correctly fails tests. | |
| stealth/space-bunny-alpha | — | closed resolved | Merged #1382, replacing grep with MATCH in four spread-test files so assertions reliably fail when expected output is absent. | |
| qwen/qwen3.6-35b-a3b | — | — | Merged updates replacing direct grep calls in spread tests with MATCH to ensure reliable test failure detection. Approved by three reviewers and modified four files. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged changes to spread tests that replace direct grep calls with MATCH, ensuring reliable test failure detection when assertions should trigger. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1514 feat(testing): add spread testing template files | charmcraft | closed | Closed unmerged as superseded by #1660 (commit 097a4e9a), which added charmcraft test and Spread templates, plus merged follow-up #1686 (58d36668). | |
| #1556 refactor(tests/spread): test with explicit charmcraft.yaml files | charmcraft | merged | Merged PR #1556 in commit 4682b32b, replacing prepare-step manipulation of charmcraft.yaml with explicit spread-test configuration files. | |
| #1555 refactor(tests/spread): stop use init in most spread tests | charmcraft | closed | Closed unmerged and replaced by PR #1556, merged as commit 4682b32b, which updated Spread tests to use explicit charmcraft.yaml files instead of charmcraft init. | |
| #1266 tests(spread): requirements dependencies | charmcraft | merged | Merged PR #1266 at 97d95b39, adding spread tests for requirements-file and charm-dependency combinations; the PR explicitly fixes #1179. | |
| #1214 tests: add more charms to large spread tests | charmcraft | closed | Closed unmerged after 1,135 days; its large-spread test expansion landed through merged PR canonical/charmcraft#1217 in commit 0f4591a9. | |
| #1322 fix(test): better matching for strict dependencies spread test | charmcraft | merged | Merged PR #1322 in commit 7c3170d3, improving the expected-output matching in the strict-dependencies spread test. | |
| #2426 fix(test): fix the reactive charm spread test | charmcraft | merged | Merged in canonical/charmcraft#2426 via commit 7c33dd1b, applying a one-line fix to the reactive-charm spread test for CHARMCRAFT-641. | |
| #1500 test(spread): replace lunar test with mantic | charmcraft | merged | Merged in canonical/charmcraft#1500 via commit 0fc289f4, replacing the Lunar spread test with Mantic in a one-line test update. | |
| #1849 tests(2.7): remove mantic spread test | charmcraft | merged | Merged PR #1849 removed the EOL Mantic spread test, restoring CI for Charmcraft 2.7 bugfix releases. | |
| #16 tests: add spread support | rockcraft | merged | Merged PR #16 added Spread support for CRAFT-865 via commit 3aeba2b3 (merge commit 5bf2680a). |