← Back to issue list

ci: run snap spread tests in one job per system

View original Github issue

Metadata

Project
craft-application
Number
#1169
Type
pull request
State
open
Author
lengau
Labels
—
Created
Updated
Closed
—

Current evaluation

Splits the snap-tests CI job into a per-system spread matrix (new spread-select job runs `spread -list openstack:`) and scopes Ubuntu Pro to 24.04. Small workflow-only diff; zizmor fails and all snap-tests jobs are still pending.

Suggested action: keep open

Reason: The PR is a coherent maintainer-authored CI refactor (+19/-5 in one workflow file) whose only clearly failing signal is zizmor, while all five snap-tests matrix jobs remain pending and the seven Copilot review comments are unresolved with no author response for 35 days. It is not closeable as stale (inactivity alone) and not ready to merge, so it stays open until the zizmor finding and the unresolved comments are addressed and the new matrix jobs actually report results.

Evaluated by: mr-cal-bot

Impact: 27 Quick Win: 7 Complexity: 32 Confidence: 68 Actionability: 38

Issue body

Splits the `snap-tests` workflow job into one job per system, dynamically selecting the systems from `spread.yaml`: - New `spread-select` job runs `spread -list openstack:` and outputs the list of systems as JSON (pattern borrowed from Charmcraft). - `snap-tests` uses a `fromJSON` matrix over those systems, running `spread openstack:<system>` in parallel jobs with `fail-fast: false`. - The Ubuntu Pro step now only runs on `ubuntu-24.04-64`, matching the pro suite's `ubuntu-24.04*` restriction in `spread.yaml`. This PR was sloperated using OpenCode and moonshotai/kimi-k3

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha
Impact: 27
Quick Win: 7
Complexity: 32
Confidence: 68
Actionability: 38
keep open Splits the snap-tests CI job into a per-system spread matrix (new spread-select job runs `spread -list openstack:`) and scopes Ubuntu Pro to 24.04. Small workflow-only diff; zizmor fails and all snap-tests jobs are still pending.
stealth/space-bunny-alpha
Impact: 28
Quick Win: 6
Complexity: 36
Confidence: 83
Actionability: 35
needs review Refactors snap Spread testing into a dynamic per-system CI matrix and scopes Ubuntu Pro testing to 24.04. Pending review: zizmor fails, all snap-tests jobs remain pending, and seven comments are unresolved.
qwen3.6-35b-a3b-mtp-q6
Impact: 40
Quick Win: 32
Staleness: 25
Complexity: 20
Confidence: 70
needs review Splits snap-tests CI workflow into one job per system using spread.yaml matrix. 0 reviewers, 7 unresolved review comments, zizmor CI failing.
qwen/qwen3.8-27b
Impact: 25
Quick Win: 18
Staleness: 5
Complexity: 30
Confidence: 80
needs review Splits the snap-tests CI job into one parallel job per openstack system via a new spread-select job and fromJSON matrix; Pro tests now run only on ubuntu-24.04-64. Fresh maintainer PR, 0 reviewers, snap-tests jobs pending, zizmor failing.
qwen/qwen3.8-27b
Impact: 45
Quick Win: 27
Staleness: 5
Complexity: 40
Confidence: 70
needs review Splits the snap-tests CI job into one job per spread system via a new spread-select job that lists openstack systems as JSON, with a fromJSON matrix and fail-fast false; Pro tests now run only on ubuntu-24.04-64. Fresh, review pending, CI running, zizmor failing.

Update history

Date Change
updated
created

Related work

  • Related To: canonical/charmcraft#2301 (confidence 82%)

    Merged charmcraft PR 'ci: split spread tests by system' (commit eec7648d) implements the same spread -list driven per-system matrix; the PR body says the pattern was borrowed from Charmcraft.

Related issues

Issue Project State Summary Similarity
#1464 ci: split spread tests by system charmcraft closed Spread test splitting from #1464 was later delivered by merged #2301 (eec7648d), adding per-system matrix jobs to reduce reruns of flaky tests.
74%