← Back to issue list

store: implement delta uploads in push.

View original Github issue

Metadata

Project
snapcraft
Number
#940
Type
pull request
State
merged
Author
squidsoup
Labels
—
Created
Updated
Closed

Current evaluation

Merged PR #940 as commit 05ba34a32, adding store-push delta uploads with SHA3-384 caching and fallback to full uploads when deltas fail or are too large.

Resolution: closed resolved

Reason: PR #940 was approved and merged as commit 05ba34a32. It implemented delta generation and upload, with full-upload fallback for generation/store failures and deltas at least 90% of the target size.

Evaluated by: mr-cal-bot

No scores available.

Issue body

LP: #1646303 A few additional notes: This branch reverts the hard linking of cache files, as the cached file needs an immutable hash and should not change when a new snap is built. A new error code, `processing_upload_delta_error` has been introduced, which the SCA notification endpoint will return in the event of a delta reconstitution failure. In the event of this error, or the client fails to generate a delta, snapcraft will fall back to the normal upload path. The branch also introduces a new dependency, pysha3, which is required for calculating sha3_384 hashes. While this can be slow across large files, a later optimisation would be skipping the source snap hash calculation once snaps are cached by their sha3_384 hash rather than revision. As a small optimisation, deltas will not be uploaded and fallback if they are 90% or over the size of the target snap. Additionally, this branch re-implements snap caching to use sha3-384 hashes exclusively, instead of parsing snap filenames for revisions which was potentially brittle.

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed resolved Merged PR #940 as commit 05ba34a32, adding store-push delta uploads with SHA3-384 caching and fallback to full uploads when deltas fail or are too large.
stealth/space-bunny-alpha — closed resolved Delta uploads for snap push were merged in canonical/snapcraft#940 (05ba34a32), adding SHA3-384 cache hashing, pysha3, and fallback for failed or oversized deltas.
qwen/qwen3.6-35b-a3b — — Merged after CI and review. Implements delta uploads for snap push, adds pysha3 dependency, switches caching to sha3-384 hashes, and includes fallback logic for failed or oversized deltas. Enabled on staging pending production rollout.
qwen3.6-35b-a3b-mtp-q6 — — Merged implementation of delta uploads for snapcraft push. Added pysha3 dependency, switched caching to sha3-384 hashes, and added fallback to full uploads on failure. Merged after staging validation and review feedback.

Update history

No update history recorded yet.

Related work

  • Related To: snapcraft#1196 (confidence 99%)

    Follow-up PR #1196 enabled delta uploads by default in commit 3b13bbd1a.

Related issues

Issue Project State Summary Similarity
#1196 store: enable delta uploads by default when pushing. snapcraft merged Merged PR #1196 enabled delta uploads by default for store pushes, resolving Launchpad #1673455; recorded in commit 3b13bbd1a.
83%
#1646303 Implement delta uploads in `push` snapcraft (launchpad) closed Delta uploads were implemented in push behind DELTA_UPLOADS_EXPERIMENTAL by merged PR #940 (commit 05ba34a32), resolving #1646303.
77%
#1780851 trying to upload a delta if delta service is disabled fails snapcraft (launchpad) closed Fixed by merged PR #2181 (commit 5437f01ca), restoring fallback to a full-snap upload when the Store rejects deltas with HTTP 501.
74%
#76 Store upload charmcraft merged Store upload was merged in PR #76 at 89adbcb3, adding the upload implementation after PR #75’s storage byte-push support.
73%