← Back to issue list

Use `snapcraft test` for spread tests

View original Github issue

Metadata

Project
craft-application
Number
#806
Type
issue
State
open
Author
lengau
Labels
Status: Triaged
Created
Updated
Closed
—

Current evaluation

Maintainer dogfooding task: switch CI from invoking spread directly to `snapcraft test`. Triaged but unimplemented — .github/workflows/spread.yaml still calls `spread ${{ matrix.spread }}`; open PR #1038 attempts it with failing CI.

Suggested action: keep open

Reason: Verified directly at HEAD that .github/workflows/spread.yaml lines 63 and 78 still run `spread ${{ matrix.spread }}` and `spread openstack:tests/spread/pro/`, and a git log search for 'snapcraft test' returns no commits, so the requested change has not landed. The issue is maintainer-authored (lengau) and labelled 'Status: Triaged', so it has been categorised and remains valid — age (449 days) is not grounds for closing. Progress exists but is stalled: PR #1038 implements exactly this ('switch from manually running spread to using snapcraft test') and has sat unreviewed for 197 days with seven CI failures spanning all craft-test jobs and snap-tests, so this issue is effectively blocked on that PR and on backend image availability (cf. closed PR #1007, which added a workaround mapping the unavailable ubuntu:26.04 image).

Evaluated by: mr-cal-bot

Impact: 42 Quick Win: 14 Complexity: 52 Confidence: 86 Actionability: 70

Issue body

### What needs to get done Instead of using spread directly, switch to using `snapcraft test` ### Why it needs to get done Dogfooding

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha
Impact: 42
Quick Win: 14
Complexity: 52
Confidence: 86
Actionability: 70
keep open Maintainer dogfooding task: switch CI from invoking spread directly to `snapcraft test`. Triaged but unimplemented — .github/workflows/spread.yaml still calls `spread ${{ matrix.spread }}`; open PR #1038 attempts it with failing CI.
stealth/space-bunny-alpha
Impact: 38
Quick Win: 13
Complexity: 55
Confidence: 80
Actionability: 78
keep open Dogfooding task: switch craft-application's CI from invoking `spread` directly to using `snapcraft test`. Still open; main's .github/workflows/spread.yaml still runs `spread openstack:` and an open PR attempts the migration.
qwen/qwen3.6-35b-a3b
Impact: 40
Quick Win: 17
Complexity: 30
Confidence: 70
Actionability: 60
keep open Switch CI to use `snapcraft test` instead of running spread directly for dogfooding. Triaged, 438 days old, with a related open PR addressing the same change.
qwen/qwen3.8-27b
Impact: 25
Quick Win: 14
Staleness: 55
Complexity: 45
Confidence: 80
Support Request: 0
keep open Dogfooding task to switch craft-application CI from running spread directly to using snapcraft test. Triaged; open PR #1038 stalled ~165 days with failing CI; workflow still runs spread directly at HEAD.
qwen/qwen3.6-35b-a3b
Staleness: 65
Complexity: 15
Confidence: 85
Support Request: 0
keep open Switch from direct spread usage to snapcraft test for dogfooding purposes. Triaged by maintainer, currently inactive with no recent technical discussion or implementation progress.
qwen3.6-35b-a3b-mtp-q6
Staleness: 55
Complexity: 30
Confidence: 75
Support Request: 0
keep open Switching spread tests to use snapcraft test for dogfooding. The issue is triaged and has been inactive for over 111 days, with an internal Jira ticket referenced.

Update history

No update history recorded yet.

Related work

  • Blocked By: craft-application#1038 (confidence 92%)

    Open PR implementing this exact change; unreviewed for 197 days with seven CI failures including all craft-test jobs and snap-tests, so the migration cannot land until those are resolved.

  • Related To: craft-application#1007 (confidence 70%)

    Closed PR adding a workaround mapping the unavailable ubuntu:26.04 backend image for Spread tests — indicates an infrastructure dependency still affecting craft-test based runs.

  • Related To: craft-application#613 (confidence 60%)

    Closed issue delivering the opt-in `test` command via PR #694 that invokes spread; this issue is the follow-up dogfooding step on top of that command.

Related issues

Issue Project State Summary Similarity
#1038 ci(spread): switch from manually running spread to using snapcraft test craft-application open Migrates dogfooding Spread runs to snapcraft test; after 197 days it remains unreviewed with seven CI failures, including all craft-test jobs and snap-tests.
77%
#6408 Re-organize spread tests snapcraft open Maintainer-filed, triaged enhancement to reorganize/dedupe Snapcraft's spread tests and drop low-value ones for faster, less flaky CI; no implementation plan or PR yet, tracked as SNAPCRAFT-1377.
74%
#1272 tests: initial setup for the snapcraft snap tests with spread snapcraft merged Merged PR #1272 as commit 545344442, adding initial Spread-based test infrastructure for the snapcraft snap.
74%
#202 Add snapcraft tests fetch-service open Snap package integration tests were added and CI now runs the Spread suite through snapcraft test, although the tracking issue remains open.
73%
#1792717 move non-mocked wiki tests to spread snapcraft (launchpad) closed Legacy wiki tests were moved to Spread in snapcore/snapcraft PR #2276, merged as commit 806b0b0db.
70%
#6510 Spread tests should run review-tools on snaps snapcraft open Enhancement to run review-tools on all spread tests that build snaps, not just manifest tests. Triaged with a contributor interested in implementing.
70%
#534 test(spread): workaround for flaky test fetch-service merged Merged PR #534 (commit 4f89bb7) worked around #533 by skipping the flaky snap names-query assertion while retaining the other spread-test checks.
70%