repo: cache stage packages for faster future use.
Metadata
Current evaluation
Merged PR #1122 (20b422459), reusing cached stage packages and copying the apt cache into cleanbuild containers to fix LP #1661383.
Resolution: closed resolved
Reason: PR #1122 was merged as commit 20b422459. It now consumes the stage-package cache, fetches packages with Version.fetch_binary(), and makes the cache available to cleanbuild.
Evaluated by: mr-cal-bot
No scores available.
Issue body
Snapcraft currently caches stage packages, but doesn't actually use them. This PR fixes LP: [#1661383](https://bugs.launchpad.net/snapcraft/+bug/1661383) by using them, and making sure cleanbuild can use the cache as well.
The way this ended up being done isn't ideal. Since `apt.Cache().fetch_archives()` seems to mangle (some) package names for no apparent reason, it's impossible to fetch stage packages with it and later determine which packages were pulled as the filenames on disk don't match up to the filenames in the archive. The settled-upon solution was to pull stage-packages using `apt.package.Version().fetch_binary()`. This works well and actually simplifies the caching mechanism, but has a few drawbacks:
- It prints directly to stdout when a package is already cached.
- Fetching stage packages one-by-one means one loses a sense of how long the overall process will actually take.
Note also that this PR **copies** the cache into the cleanbuild container as opposed to bind-mounting read-only. This was done so the caching code was more robust. Otherwise it would need to account for being unable to do the following things:
- Being unable to clean the cache when sources changes
- Being unable to update the index
- Being unable to fetch packages
And have backup plans. Such code would be complex and difficult to test, and it seemed it wasn't worth it for a relatively small efficiency gain, particularly considering that ideally the cache would be editable from the container anyway, that just doesn't seem possible today without modifications to /`etc/subgid`. Of course this is up for debate!
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Merged PR #1122 (20b422459), reusing cached stage packages and copying the apt cache into cleanbuild containers to fix LP #1661383. | |
| stealth/space-bunny-alpha | — | closed resolved | Merged PR #1122 (commit 20b422459), fixing LP bug #1661383 by reusing cached stage packages and copying the cache into cleanbuild containers. | |
| qwen/qwen3.6-35b-a3b | — | — | Merged to fix Snapcraft ignoring cached stage packages. The implementation uses fetch_binary for reliable retrieval and copies the cache into cleanbuild containers for robustness. Approved by two reviewers. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged to resolve LP #1661383, enabling Snapcraft to utilize cached stage packages for faster builds. The implementation uses apt.package.Version().fetch_binary() and copies the cache into cleanbuild containers for robustness. |
Update history
No update history recorded yet.
Related work
-
Related To:
canonical/snapcraft#1216
(confidence 98%)
Later changed cleanbuild to avoid copying the cache into the container.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1113 repo: cache stage packages for faster future use. | snapcraft | closed | Closed unmerged in favor of #1122, which merged the stage-package caching and cleanbuild improvements as commit 20b422459. | |
| #1661383 stage package cache no longer working | snapcraft (launchpad) | closed | Stage-package cache reuse was restored: snapcraft PR #1122 (commit 20b422459, 'repo: cache stage packages for faster future use') merged, reusing cached debs and copying the apt cache into cleanbuild containers, closing LP #1661383. | |
| #3110 repo: use persistent cache for fetching stage packages | snapcraft | closed | PR #3110 (persistent cache for fetching stage packages) was closed unmerged with 0 reviews and 1 open maintainer question. Its dependency #3109 also closed unmerged; the feature was later delivered by merged PR #3585 (CRAFT-488) using host /var/lib/apt state. | |
| #3030 repo: drop _AptCache and add migrate to install_stage_packages() | snapcraft | merged | Merged in PR #3030 as commit 66467629d, replacing custom _AptCache handling with install_stage_packages() and updating package staging, tests, and ROS plugins. | |
| #3086 repo: restore marked-install strategy for apt-cache | snapcraft | merged | Merged PR #3086 (17cc75aae), restoring AptCache marked installs for build and stage packages and fixing primed-stage-packages name/version parsing from ':' to '='. | |
| #118 packages/apt: use host state for apt cache (CRAFT-487) | craft-parts | merged | Merged PR #118 unified stage and host APT caches using /var/lib/apt, removed the update interface, and made stage package-list refresh a no-op. | |
| #2119 repo: automatically prune unneeded stage-packages | snapcraft | merged | Merged as b9faf80bd, adding APT cache analysis to remove automatically installed stage-packages left unneeded after alternative dependency resolution. | |
| #3585 repo: use host state for apt cache (CRAFT-488) | snapcraft | merged | Merged #3585 via f817e34f2 (merge 0ae235a09), replacing split stage APT state with the host cache at /var/lib/apt for CRAFT-488. |