fix(tests/spread): fix reactive spread tests
Metadata
Current evaluation
Merged #2763 as 6e775f4a: split reactive spread coverage by legacy/Resolute, enabled binary wheels, and removed obsolete build flags.
Resolution: closed resolved
Reason: PR #2763 was approved and merged as commit 6e775f4a. It replaced the shared reactive test with legacy and Resolute variants and corrected charm-build dependency options.
Evaluated by: mr-cal-bot
No scores available.
Issue body
This PR fixes the failing reactive spread tests by making several structural and configuration changes:
1. **Test Splitting**: The single `reactive` test has been split into two distinct tests to avoid complex conditionals and cross-talk:
- `reactive-legacy`: Runs on Ubuntu 20.04 and 22.04, testing the `2.x/stable` and `stable` charm channels. It dynamically sets its base channel to match the host system's base.
- `reactive-resolute`: A standalone test specifically for Ubuntu 26.04, configured to strictly test the `charm/latest/stable` channel.
2. **Dependency Resolution Fix**: Explicitly adds `--binary-wheels` to the `charm-build` arguments in `charmcraft.yaml`. This prevents `charm-build` from defaulting to source builds (which fail due to PEP-517 isolated environments downloading newer `setuptools` versions that lack `pkg_resources`). Using binary wheels avoids these source build failures and speeds up the tests.
3. **Cleanup**: Removes `--upgrade-buildvenv-core-deps` and `--ignore-requires-python` flags from all `charmcraft.yaml` templates, as they were contributing to dependency resolution failures on older versions and are no longer necessary with binary wheels.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Merged #2763 as 6e775f4a: split reactive spread coverage by legacy/Resolute, enabled binary wheels, and removed obsolete build flags. | |
| stealth/space-bunny-alpha | — | closed resolved | Merged #2763: split reactive spread tests into legacy and Resolute variants, enabled binary wheels, and removed obsolete build flags to fix dependency-resolution failures. | |
| qwen/qwen3.6-35b-a3b | — | — | Merged after resolving failing reactive spread tests. The changes split the test suite into legacy and resolute variants, switched to binary wheels to fix dependency resolution, and removed obsolete flags. Approved by reviewers and integrated. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
5
Complexity:
15
Confidence:
90
|
needs review | Removes unsupported charm-build flags from reactive charmcraft.yaml template to fix spread tests on older snap channels, with dynamic injection for 26.04. Currently awaiting review. |
Update history
| Date | Change |
|---|---|
| updated |
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #2762 tests/spread: fix reactive spread tests on older channels | charmcraft | closed | Closed unmerged after 86 days; no resolving commit or replacement PR was identified, and the Conventional Commits policy check remained failing. | |
| #4433 tests(spread): add remote-build spread test | snapcraft | merged | Merged #4433 (commit 4a0a91117), adding weekly spread coverage for new and legacy remote-build implementations and resolving #4398. | |
| #1366 tests(spread): make release test less flaky | charmcraft | merged | Merged #1366 at commit 05b18a03, reducing flakiness in the Spread release test and resolving #1362. | |
| #3432 spread tests: remove legacy plugin tests | snapcraft | merged | PR #3432 (commit 950fa2508, f2c9caa04) merged: deleted 665 lines of legacy plugin spread tests across 29 files, removing outdated plugin coverage without affecting code coverage (90.83%). | |
| #5297 test: re-enable devel spread tests | snapcraft | merged | Merged in #5297 (commit 65a6004f5), re-enabling and simplifying devel spread tests, adding a second test, and resolving #4921. | |
| #6350 test: fix spread tests | snapcraft | merged | Merged PR #6350 as a18c7f485, fixing all non-store Spread tests and OSV scanning, including pinning Black 26.3.1 to avoid patchelf failures. | |
| #291 Refactor spread testing to use spread extensions | fetch-service | closed | Fixed by merged PR #299 (commit 44d9906), which refactored spread tests to generate spread.yaml and use the new spread extension framework. | |
| #1287 tests: minor cleanups on the spread tests | snapcraft | merged | Merged into master as commit 1cf0f8c9f "tests: minor cleanups on the spread tests (#1287)" — a test-only tidy-up touching 4 spread test files (+16/-6), reviewed and approved by fgimenez, with coverage unchanged at 95.27%. | |
| #4398 Add spread test for remote-build | snapcraft | closed | Remote-build Spread coverage was added in commit 4a0a91117, resolving the missing tests for the new builder and legacy fallback. | |
| #1600 test(spread): move flaky parallel-build test to weeklies | charmcraft | merged | Merged in commit a1d285c7, moving the flaky parallel-build Spread test from regular CI to weeklies as a workaround for #1332. |