Improve integration tests
Metadata
Current evaluation
Enhancement proposal to restructure integration tests, either by migrating to spread or reorganizing existing tests to address disk space exhaustion on GitHub runners.
Suggested action: keep open
Reason: This is a valid enhancement from maintainer mr-cal about improving test infrastructure to address disk space exhaustion on GitHub runners. The issue has no comments and hasn't been actively worked on, but the underlying problem (integration tests consuming 99% of runner disk space) is likely still relevant. The complexity is high as it involves architectural decisions (spread migration vs. reorganization) and requires planning.
Impact:
60
Quick Win:
18.0
Staleness:
65
Complexity:
70
Confidence:
60
Support Request:
10
Issue body
### What needs to get done
## Option 1 - spread
Move the integration tests to spread. This includes:
- Create an entrypoint (see [craft-parts/main.py](https://github.com/canonical/craft-parts/blob/cd3a00787f13fb0b046dbb298c150bde41b76766/craft_parts/main.py)) or a wrapper script
- Set up spread infrastructure (github secrets, workflows, spread.yaml)
- Translate integration tests to spread tests
## Option 2 - improve existing integration tests
Ideas:
- Run the integration tests and check if any lxd instances or images are not being cleaned up
- Can we parametrize or re-organize the tests?
- For example, can the integration tests run by base (all the `ubuntu` integration tests run and get cleaned, then `almalinux`, then `centos`)
Related - # 381 discusses how the integration tests are disorganized and lacking. We may need to dedicate time to research and plan the best path forward.
### Why it needs to get done
The integration tests use about 99% of the available disk space of the github runners. #330 adds another lxd image and now the integrations cannot run.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen3.6-35b-a3b-mtp-q6 |
Impact:
60
Quick Win:
18.0
Staleness:
65
Complexity:
70
Confidence:
60
Support Request:
10
|
keep open | Enhancement proposal to restructure integration tests, either by migrating to spread or reorganizing existing tests to address disk space exhaustion on GitHub runners. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
65
Complexity:
75
Confidence:
85
Support Request:
0
|
needs triage | Proposal to restructure or migrate integration tests to resolve GitHub runner disk space exhaustion and improve test organization. Currently awaiting team discussion and implementation path. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
75
Complexity:
75
Confidence:
65
Support Request:
10
|
needs triage | Enhancement proposal to restructure or migrate integration tests to address GitHub runner disk space exhaustion. Open for ~3 years with no concrete implementation plan or assignee. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
80
Complexity:
70
Confidence:
75
Support Request:
5
|
needs triage | Enhancement proposal to restructure CI integration tests to reduce GitHub runner disk usage. Author suggests migrating to spread infrastructure or refactoring existing tests. Open for ~2.7 years with no maintainer discussion or prioritization. |
Update history
No update history recorded yet.
Related work
-
Duplicate Of:
craft-providers#381
(confidence 100%)
The issue body references #381 as a related issue, but this issue IS craft-providers#381 (same title, same content).
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #381 Improve integration tests | craft-providers | open | Enhancement proposal to restructure integration tests, add timing, and create new test suites for bases verification. Open, awaiting implementation. | |
| #665 Start using `spread` for integration tests | charmcraft | closed | Completed and merged via pull request #862, which implemented the spread infrastructure and added integration tests. The issue was closed after the changes were successfully integrated. | |
| #6408 Re-organize spread tests | snapcraft | open | Enhancement to reorganize spread test directories, deduplicate tests, and remove unnecessary spread tests for faster CI. Triaged by maintainer, 8 days old. |