← Back to issue list

Warn on validation set update without incrementing sequence number

View original Github issue

Metadata

Project
snapcraft
Number
#5367
Type
issue
State
closed
Author
ernestl
Labels
Status: Triaged Type: Enhancement
Created
Updated
Closed

Current evaluation

Added the validation-set sequence warning in merged PR #5535 (commit f012b667e).

Resolution: closed resolved

Reason: PR #5535 resolved the issue in commit f012b667e. It warns when an edited validation set does not increment its sequence and allows the assertion to be amended before submission.

Evaluated by: mr-cal-bot

No scores available.

Issue body

### What needs to get done Issue a warning when snapcraft edit-validation-sets <account-id> <set-name> <sequence> is used to modify an assertion without incrementing the sequence number. **BEHAVIOR:** The warning should highlight that: - if anything goes wrong with an intended snap refresh, snap will not automatically revert to a valid state, the validation set state will become and remain invalid. - if the validation set specifies e.g. a revision that exists in the store and is not available on the architecture it will still refresh the validation set and become invalid without warning. Perhaps something like: `WARNING: not incrementing the sequence number means invalid changes or snap refresh failure will invalidate this assertion with no means to automatically revert to a valid state` Provide the opportunity to edit again, as is already the case with store error **APPROACH:** I am aware this can in theory be solved by the store as well, leaving it up to snapcraft team to decide on preferred approach. ### Why it needs to get done This request stems from an investigation into unexpected behavior observed by an early adopter of the validation-sets feature, as reported in SF. The outcome led us to re-evaluate the implications of making changes within incrementing assertion sequence number: When a device is pinned to an enforced and valid validation set at sequence <n>, subsequent mutation of the SAME SEQUENCE via `snapcraft edit-validation-sets <account-id> <set-name> <n>`, on refresh, can cause transition to invalid state due to things like: - refresh failure combined with inability to revert due to non-distinct sequence - revision of the correct snap but for a different architecture which is not store checked during submission The general idea/promise with enforced pinned validation sets are: `An assertion will move to the latest sequence if present in the store and if the installed snaps, including any newer revisions in the store, still satisfy their respective validation set assertions.` (https://snapcraft.io/docs/validation-sets) So at a sequence level it guards against actions that will cause transition to invalid. This is not the case with sub-sequence changes, which presents the real danger of accidentally invalidating a managed fleet.

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed resolved Added the validation-set sequence warning in merged PR #5535 (commit f012b667e).
stealth/space-bunny-alpha — closed resolved Added an edit warning when a validation set is changed without incrementing its sequence; implemented by merged PR #5535 (commit f012b667e).
qwen/qwen3.6-35b-a3b — — Implemented via PR #5535. Added a client-side warning prompt when editing a validation set without incrementing the sequence number to prevent accidental fleet invalidation. Changes are available in the latest/edge channel.
qwen/qwen3.6-35b-a3b — — Implemented via PR #5535. Added a client-side warning prompt when editing validation sets without incrementing the sequence number to prevent accidental fleet invalidation. The fix is available in the latest/edge/pr-5535 channel.
qwen3.6-35b-a3b-mtp-q6 — — Resolved via PR #5535. Added a client-side warning when updating validation sets without incrementing the sequence number, preventing accidental fleet invalidation. The fix is available on the latest/edge channel.

Update history

No update history recorded yet.

Related work

  • Likely Fixed By: canonical/snapcraft#5535 (confidence 100%)

    Merged PR implementing the requested sequence validation and retryable warning.

Related issues

Issue Project State Summary Similarity
#5545 fix(validation-sets): allow users to ignore sequence increment warning snapcraft merged Merged PR #5545 (cb96b3533), allowing users to ignore validation-set sequence-increment warnings and still submit assertions, fixing the overlooked scenario in #5535.
76%
#5535 feat(validation-sets): warn when the sequence isn't incremented snapcraft merged PR #5535 by mr-cal merged (commit f012b667e): adds a SnapcraftAssertionWarning when a validation set's sequence isn't incremented, plus shared _validate_assertion infrastructure. Fixes #5367.
73%