Mark chisel tests as flaky
Metadata
Current evaluation
Chisel integration tests were marked flaky via merged PR #1308 (commit ec02157c), adding retries for intermittent archive.ubuntu.com failures.
Resolution: closed resolved
Reason: Merged PR #1308 resolved the issue in commit ec02157c by marking the network-dependent Chisel integration tests flaky so transient failures can be retried.
Evaluated by: mr-cal-bot
No scores available.
Issue body
### What needs to get done
The integration tests for chisel are dependent on perfect network connections between the device and `archive.ubuntu.com`, which isn't reasonable.
https://github.com/pytest-dev/pytest-rerunfailures
### Why it needs to get done
We should allow them to be retried at least once before failing in order to avoid re-running the entire workflow just because those tests failed.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Chisel integration tests were marked flaky via merged PR #1308 (commit ec02157c), adding retries for intermittent archive.ubuntu.com failures. | |
| stealth/space-bunny-alpha | — | closed resolved | Chisel integration tests were marked flaky by merged PR #1308 (ec02157c), allowing one retry to handle intermittent archive.ubuntu.com failures. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without implementation. The enhancement to retry flaky chisel tests via pytest-rerunfailures was not applied. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without implementation. The enhancement to retry flaky chisel tests using pytest-rerunfailures was logged but not merged, likely abandoned or superseded. | |
| qwen/qwen3.6-35b-a3b | — | — | Abandoned. A Jira ticket was created to track the enhancement for retrying flaky chisel tests, but no code changes or PRs were merged. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without implementation. The enhancement to retry flaky chisel integration tests was superseded by an internal Jira ticket. No code changes or pull requests were merged. |
Update history
No update history recorded yet.
Related work
-
Likely Fixed By:
canonical/craft-parts#1308
(confidence 100%)
Merged PR #1308 implemented the flaky-test retries in commit ec02157c.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1308 fix(test): mark chisel integration tests as flaky | craft-parts | merged | Merged as ec02157c, marking network-sensitive Chisel integration tests as flaky with retries; fixes #1306. |