test: add retry on reactive tests
Metadata
Current evaluation
Adds retry logic to flaky reactive tests to reduce test failures. Small change (+6/-2 lines) approved by 1 reviewer, but some CI checks are failing.
Suggested action: needs review
Reason: PR is approved by 1 reviewer but some CI checks are failing (snap-tests on ubuntu-20.04-64, snap-store-tests, snap-tests on ubuntu-22.04-64, snap-tests on ubuntu-25.10-64). The change is small and straightforward, but failing CI needs investigation before merging.
Impact:
20
Quick Win:
18.0
Staleness:
5
Complexity:
10
Confidence:
75
Issue body
The reactive tests are rather flaky. While they don't do anything particularly special (they just pack a charm, as do many other spread tests), what sets them apart is they are also remarkably slow and so can run up against `kill-timeout`s often too.
This isn't a very great long-term change, but it's a pilot solution that we can look to to inform our decision on rerun mechanisms in our spread tests later.
Across 5 runs (the initial 3 + 2 reruns of the 20.04 variant), the reactive tests never failed.
CHARMCRAFT-712
---
- [ ] I've followed the [contribution guidelines](https://github.com/canonical/charmcraft/blob/main/CONTRIBUTING.md).
- [ ] I've signed the [CLA](http://www.ubuntu.com/legal/contributors/).
- [ ] I've successfully run `make lint && make test`.
- [ ] I've added or updated any relevant documentation.
- [ ] In documents I changed, I [added a meta description](https://canonical-starflow.readthedocs-hosted.com/how-to/add-a-page-meta-description/) if one was missing.
- [ ] I've updated the relevant release notes.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b |
Impact:
20
Quick Win:
18.0
Staleness:
5
Complexity:
10
Confidence:
75
|
needs review | Adds retry logic to flaky reactive tests to reduce test failures. Small change (+6/-2 lines) approved by 1 reviewer, but some CI checks are failing. | |
| qwen/qwen3.6-35b-a3b |
Impact:
25
Quick Win:
21.25
Staleness:
5
Complexity:
15
Confidence:
70
|
needs review | Adds retry logic to flaky reactive spread tests to mitigate kill-timeout issues. Small change (+6/-2 lines) with some CI checks failing on specific Ubuntu variants. | |
| qwen/qwen3.6-35b-a3b |
Impact:
35
Quick Win:
31.5
Staleness:
5
Complexity:
10
Confidence:
65
|
needs review | Adds retry logic to reactive spread tests to address flakiness. Fresh PR with 0 reviewers, 0 comments, and several failing CI checks. | |
| qwen/qwen3.6-35b-a3b |
Impact:
30
Quick Win:
25.5
Staleness:
0
Complexity:
15
Confidence:
70
|
needs review | Adds retry logic to reactive spread tests to address flakiness and kill-timeout issues. New PR with 0 reviews, CI still running. |
Update history
| Date | Change |
|---|---|
| updated | |
| updated | |
| updated | |
| updated | |
| created |
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #983 test: make retry unit test more deterministic | craft-providers | merged | Merged. Changes refactor a retry unit test for improved determinism. Approved by a reviewer, passed CI checks, and updated one file (+58/-9). |