← Back to issue list

Improve integration tests

View original Github issue

Metadata

Project
craft-providers
Number
#430
Type
issue
State
open
Author
mr-cal
Labels
Type: Enhancement
Created
Updated
Closed
—

Current evaluation

Integration tests remain disk-bound: LXD images consume nearly all GitHub runner storage, and the proposed Spread migration or pytest cleanup/parametrization overhaul is still unresolved.

Suggested action: keep open

Reason: The maintainer-authored issue reports that integration tests consume about 99% of runner disk and that #330's additional LXD image prevents them from running. The repository still has pytest integration tests and QA workflows without a Spread suite, although a CLI entrypoint and split LXD/macOS jobs now exist; no maintainer decision or acceptance criteria establishes that the overall work is complete.

Evaluated by: mr-cal-bot

Impact: 72 Quick Win: 11 Complexity: 78 Confidence: 84 Actionability: 68

Issue body

### What needs to get done ## Option 1 - spread Move the integration tests to spread. This includes: - Create an entrypoint (see [craft-parts/main.py](https://github.com/canonical/craft-parts/blob/cd3a00787f13fb0b046dbb298c150bde41b76766/craft_parts/main.py)) or a wrapper script - Set up spread infrastructure (github secrets, workflows, spread.yaml) - Translate integration tests to spread tests ## Option 2 - improve existing integration tests Ideas: - Run the integration tests and check if any lxd instances or images are not being cleaned up - Can we parametrize or re-organize the tests? - For example, can the integration tests run by base (all the `ubuntu` integration tests run and get cleaned, then `almalinux`, then `centos`) Related - # 381 discusses how the integration tests are disorganized and lacking. We may need to dedicate time to research and plan the best path forward. ### Why it needs to get done The integration tests use about 99% of the available disk space of the github runners. #330 adds another lxd image and now the integrations cannot run.

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha
Impact: 72
Quick Win: 11
Complexity: 78
Confidence: 84
Actionability: 68
keep open Integration tests remain disk-bound: LXD images consume nearly all GitHub runner storage, and the proposed Spread migration or pytest cleanup/parametrization overhaul is still unresolved.
stealth/space-bunny-alpha
Impact: 35
Quick Win: 2
Complexity: 82
Confidence: 70
Actionability: 32
keep open Enhancement to overhaul integration tests: migrate to Spread or improve pytest cleanup/parametrization, after LXD images exhausted runner disk. CI test-splitting landed; Spread migration and disabled multipass/macOS jobs still open.
stealth/space-bunny-alpha
Impact: 58
Quick Win: 10
Complexity: 68
Confidence: 82
Actionability: 52
keep open Open maintainer enhancement to overhaul integration tests: migrate to Spread or improve pytest cleanup and parametrization, prompted by LXD images consuming nearly all GitHub runner disk and blocking test runs.
qwen3.6-35b-a3b-mtp-q6
Impact: 60
Quick Win: 18
Staleness: 65
Complexity: 70
Confidence: 60
Support Request: 10
keep open Enhancement proposal to restructure integration tests, either by migrating to spread or reorganizing existing tests to address disk space exhaustion on GitHub runners.
qwen/qwen3.6-35b-a3b
Staleness: 65
Complexity: 75
Confidence: 85
Support Request: 0
needs triage Proposal to restructure or migrate integration tests to resolve GitHub runner disk space exhaustion and improve test organization. Currently awaiting team discussion and implementation path.
qwen3.6-35b-a3b-mtp-q6
Staleness: 75
Complexity: 75
Confidence: 65
Support Request: 10
needs triage Enhancement proposal to restructure or migrate integration tests to address GitHub runner disk space exhaustion. Open for ~3 years with no concrete implementation plan or assignee.

Update history

No update history recorded yet.

Related work

  • Related To: craft-providers#381 (confidence 98%)

    The issue explicitly links #381 as related discussion of disorganized and inadequate integration tests.

  • Caused By: craft-providers#330 (confidence 93%)

    The issue identifies #330 as adding another LXD image; that merged Ubuntu 23.10 support increased the integration-test resource demand.

Related issues

No related issues found above the similarity threshold.