`test` command always builds with Ubuntu 24.04
Metadata
Current evaluation
`snapcraft test` on a core22 snap silently builds with Ubuntu 24.04 instead of erroring. Triaged by maintainers; PR #6450 (fail early for core22) was closed unmerged, so the bug is still present in the tree.
Suggested action: keep open
Reason: Confirmed the bug is live at HEAD: snapcraft/cli.py:60 registers `craft_application.commands.lifecycle.TestCommand` with no snapcraft subclass or base guard (grep for `class TestCommand` finds nothing outside an unrelated test class), so core22 projects fall into the craft-application path and build on 24.04. The issue is triaged (labels 'Status: Triaged', 'Type: Bug') and mr-cal proposed the exact fix, but PR #6450 was closed without a merge commit, so nothing resolved it. Keep open; the fix is a small subclass raising `errors.FeatureNotImplemented` mirroring `TryCommand._run` (snapcraft/commands/lifecycle.py:261-264) plus a unit test. Confidence is capped because the tool output gives no reason why #6450 was closed, which may indicate a blocker not captured here.
Evaluated by: mr-cal-bot
Issue body
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha |
Impact:
48
Quick Win:
31
Complexity:
28
Confidence:
82
Actionability:
90
|
keep open | `snapcraft test` on a core22 snap silently builds with Ubuntu 24.04 instead of erroring. Triaged by maintainers; PR #6450 (fail early for core22) was closed unmerged, so the bug is still present in the tree. | |
| stealth/space-bunny-alpha |
Impact:
40
Quick Win:
28
Complexity:
18
Confidence:
88
Actionability:
85
|
keep open | Bug: `snapcraft test` is a craft-application command, so it builds core22 snaps in an Ubuntu 24.04 environment. Triaged by a maintainer; fix PR #6450 (reject core22 early) is open awaiting review. | |
| qwen/qwen3.6-35b-a3b |
Impact:
70
Quick Win:
39
Complexity:
30
Confidence:
85
Actionability:
80
|
keep open | The `test` command is only in CORE24_LIFECYCLE_COMMAND_GROUP but core22 snaps can still invoke it, causing it to build with Ubuntu 24.04 instead of core22. Maintainer suggested overriding TestCommand to check base and raise an error. | |
| qwen/qwen3.8-27b |
Impact:
50
Quick Win:
35
Staleness:
55
Complexity:
30
Confidence:
75
Support Request:
0
|
keep open | test command always builds with Ubuntu 24.04 for core22 snaps because it's a craft-application command not in the core22 lifecycle group; triaged with a suggested fix to raise an error for core22 base, but no fix has been implemented yet | |
| qwen/qwen3.6-35b-a3b |
Staleness:
85
Complexity:
15
Confidence:
75
Support Request:
0
|
close stale | Bug where the `test` command forces Ubuntu 24.04 builds for core22 snaps. Triaged with a suggested fix, but inactive for over 14 months. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
85
Complexity:
20
Confidence:
85
Support Request:
10
|
keep open | Test command forces Ubuntu 24.04 builds, breaking core22 snaps. Triaged with a proposed base-check fix, but inactive for over a year. |
Update history
| Date | Change |
|---|---|
| updated | |
| updated |
Related work
-
Related To:
snapcraft#6450
(confidence 80%)
PR 'fix(test): error for core22 snaps' implements the approach mr-cal suggested in the comments, but is closed with no merge commit; verified the guard is absent from snapcraft/cli.py and snapcraft/commands/lifecycle.py at HEAD, so the fix did not land.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #6450 fix(test): error for core22 snaps | snapcraft | closed | PR #6450 proposed an early core22 error for snapcraft test, but was closed unmerged with review pending; #5479 remains open. |