← Back to issue list

fix: wait when retrying snap store; fail install loudly

View original Github issue

Metadata

Project
craft-parts
Number
#1632
Type
pull request
State
open
Author
james-garner-canonical
Labels
—
Created
Updated
Closed
—

Current evaluation

Fixes silent snap install failures when store queries timeout by removing the store-query gate and adding retries with sleep. Awaiting review; lint check failing, 2 unresolved review comments.

Suggested action: needs review

Reason: The PR addresses a real bug where transient snap store query failures cause silent install skips, surfacing only as cryptic validation errors later. However, it has 2 unresolved review comments and a failing lint check that need to be addressed before merge.

Evaluated by: mr-cal-bot

Impact: 55 Quick Win: 16 Complexity: 25 Confidence: 75 Actionability: 40

Issue body

This PR aims to address some unusual `charmcraft pack` failures we've seen in CI, for example [in the charmlibs monorepo](https://github.com/canonical/charmlibs/actions/runs/28564481582/job/84688902250) and [with the Ubuntu charm](https://github.com/canonical/charm-ubuntu/actions/runs/28907087457/job/85756388863). The relevant output is: ``` + charmcraft pack Launching managed ubuntu 24.04 instance... Creating new instance from remote Creating new base instance from remote Creating new instance from base instance Starting instance Initializing lifecycle Installing build-packages Installing build-snaps Environment validation failed for part 'charm': 'uv' not found and part 'charm' does not depend on a part named 'uv-deps' that would satisfy the dependency. Failed to run charmcraft in instance Full execution log: '...' ``` An AI-powered investigation points to the culprit being in `craft-parts`, where `install_snaps()` gates the `snap install` call behind a successful snapd `/v2/find` store query; when that query fails transiently, `get_store_snap_info()` exhausts 5 zero-delay retries and returns `None` instead of raising, so the build-snap is silently never installed and the failure only surfaces later as a missing command. This PR makes two changes: 1. Don't gate the `snap install` call on the store query, so if the store is unavailable the install will fail loudly rather than surfacing only cryptically at validation time. 2. Try to make querying the store less likely to fail by adding a sleep between the retries, which would otherwise be performed back-to-back. We don't gate the `snap install` because: - Original snap support ([`da71b734`, 2021](https://github.com/canonical/craft-parts/commit/da71b73409330c049357fe2bc99f5c65327c5713)) made the install decision independently of the store query, failing loudly with a type error. - [`8aeb3ea8` — "packages: address issues raised by pyright 1.1.151"](https://github.com/canonical/craft-parts/commit/8aeb3ea8e32edccf04b47bae95eac2bface135c4) wrapped the whole block (including `snap_pkg.install()`) in `if store_snap_info:` to silence a None-indexing warning (the previous loud failure). This changed the behaviour on store query failure to silently skip installation. A sleep seems like a reasonable approach, because: - craft-providers restarts snapd during instance setup and guards the following API calls with a retry loop that waits between attempts, commented with: [`base.py`: "snapd claims it's ready but actually isn't (SNAPDENG-36387). The workaround is to retry."](https://github.com/canonical/craft-providers/blob/3.3.0/craft_providers/base.py#L625-L632) - craft-parts own snapd query retry loop ([`get_store_snap_info`](https://github.com/canonical/craft-parts/blob/5363f3fda4fb004e038eedaf012938814beee041/craft_parts/packages/snaps.py#L119-L144)) fires all 5 attempts back-to-back with no delay. - A sleep between retries is used elsewhere in craft-parts for CI-transient failures, e.g. the umount retry in [`utils/os_utils.py`](https://github.com/canonical/craft-parts/blob/5363f3fda4fb004e038eedaf012938814beee041/craft_parts/utils/os_utils.py#L259-L286) ("unmount in Github CI fails randomly and needs a retry", `time.sleep(1)` between attempts). --- - [x] Have you followed the guidelines for contributing? - [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)? - [x] Have you successfully run `make lint && make test`? - [x] Have you added an entry to the changelog (`docs/reference/changelog.rst`)?

Evaluation history

Date Model Scores Action Summary
qwen/qwen3.6-35b-a3b
Impact: 55
Quick Win: 16
Complexity: 25
Confidence: 75
Actionability: 40
needs review Fixes silent snap install failures when store queries timeout by removing the store-query gate and adding retries with sleep. Awaiting review; lint check failing, 2 unresolved review comments.
stealth/space-bunny-alpha
Impact: 58
Quick Win: 17
Complexity: 30
Confidence: 78
Actionability: 42
needs review Removes the store-query gate around snap install in install_snaps() so failed store lookups fail loudly, and adds a sleep between the 5 snapd /v2/find retries. Well-reasoned CI-flakiness fix, but lint is red and it awaits first review.
stealth/space-bunny-alpha
Impact: 55
Quick Win: 18
Complexity: 35
Confidence: 80
Actionability: 50
needs review Stops gating `snap install` on the snapd store query in install_snaps() and adds a sleep between the 5 store-query retries, so transient store failures fail loudly instead of silently skipping build-snaps. Lint/files check failing, 2 review threads unresolved, no reviewers yet.
stealth/space-bunny-alpha
Impact: 45
Quick Win: 14
Complexity: 28
Confidence: 93
Actionability: 44
needs review Adds retry delays to snap-store lookups and makes snap installation independent of store-query success to surface failures earlier. No reviews yet; lint fails, two review threads are unresolved, and many checks are pending.
qwen3.6-35b-a3b-mtp-q6
Impact: 60
Quick Win: 33
Staleness: 70
Complexity: 45
Confidence: 65
keep open Fixes snap store query failures by adding retry delays and removing the store-query gate on snap install to fail loudly instead of silently skipping
qwen/qwen3.6-35b-a3b
Staleness: 15
Complexity: 30
Confidence: 85
needs review Fixes transient snap store query failures causing cryptic build errors by adding delays between retries and removing the silent skip gate. Currently awaiting maintainer review with 2 unrelated CI failures.
qwen3.6-35b-a3b-mtp-q6
Staleness: 5
Complexity: 20
Confidence: 80
needs review Fixes snap store retry logic to wait properly and fail loudly on install errors. Currently awaiting CI checks and maintainer review.

Update history

Date Change
updated

Related issues

Issue Project State Summary Similarity
#1739 fix: fail fast when snap store retries are exhausted for build-snaps craft-parts merged Merged PR #1739; re-raises final non-404 HTTPError after snap store retry exhaustion, preventing silent build-snaps installation skip and surfacing real store failure.
70%