test: randomize the order of tests
Metadata
Current evaluation
Merged a pilot implementation to randomize test execution order, preventing inter-test dependencies. Approved by two reviewers with all CI checks passing. Diff adds 16 lines and removes 1 across two files.
Suggested action: —
No scores available.
Issue body
- [ ] Have you followed the guidelines for contributing?
- [ ] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [ ] Have you successfully run `make lint && make test`?
---
In order to avoid inter-test dependencies, tests should be randomized. This is a pilot PR to see how easy this would be to adopt into all craft repositories.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Merged a pilot implementation to randomize test execution order, preventing inter-test dependencies. Approved by two reviewers with all CI checks passing. Diff adds 16 lines and removes 1 across two files. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged a pilot change to randomize test execution order, eliminating inter-test dependencies. The update establishes a reusable testing pattern for broader adoption across craft repositories. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged a pilot pull request to randomize test execution order, eliminating inter-test dependencies. This establishes a reusable testing pattern for adoption across all craft repositories. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #289 test: randomize the order of tests | craft-store | merged | Merged after two approvals and passing CI. Changes modify two files to randomize test execution order, preventing order-dependent failures. The update adds 1076 lines and removes 1061 lines. | |
| #461 test: improve test coverage | fetch-service | merged | Merged to improve test coverage, adding 270 lines across three files. Approved by two reviewers with all CI checks passing. | |
| #1928 tests: remove duplicate tests | snapcraft | merged | Merged changes that remove duplicate tests and clarify test base class names to prevent future duplication. Approved by two reviewers, passed CI, and reduced unit test execution time. | |
| #458 test: improve test coverage | fetch-service | merged | Merged after approval from two reviewers and passing CI checks. Added 370 lines of test code across five files to improve overall test coverage. | |
| #278 tests: additional tests to improve coverage | fetch-service | merged | Merged after two reviews and passing CI checks. Adds additional tests across ten files to improve code coverage, changing +386/-60 lines. | |
| #983 test: make retry unit test more deterministic | craft-providers | merged | Merged. Changes refactor a retry unit test for improved determinism. Approved by a reviewer, passed CI checks, and updated one file (+58/-9). |