← Back to issue list

perf(state): cache source outdated checks within planning

View original Github issue

Metadata

Project
craft-parts
Number
#1757
Type
pull request
State
open
Author
chris-bainbridge
Labels
—
Created
Updated
Closed
—

Current evaluation

Adds planning-scoped cache for source outdated checks in StateManager, keyed by part and pull-state serial, with defensive report copies. ~96% planning speedup on large SDK. CI failing lint/files and OSV-scanner; one unresolved review comment.

Suggested action: keep open

Reason: The PR reports a 96% planning-time reduction and adds a scoped cache with defensive copies, but CI is failing lint/files and Security scan / OSV-scanner, and there is one unresolved review comment, so it needs author follow-up before maintainer review.

Evaluated by: mr-cal-bot

Impact: 75 Quick Win: 12 Complexity: 60 Confidence: 60 Actionability: 40

Issue body

Planning a lifecycle repeatedly asks whether a part's source is outdated. Each check scans the local source tree, and during `Sequencer.plan` the same part is queried many times as the dependency graph is traversed. For large local source trees, such as a multi-gigabyte SDK, these repeated scans dominate planning time. Memoize source outdated checks for the duration of a single `Sequencer.plan` call. Results are keyed by part and pull-state serial, so repeated planning checks reuse the earlier result instead of rescanning the tree. Because the serial changes whenever the state changes, a state update cannot produce a stale cached result. The cache is scoped strictly to planning: - Direct `StateManager` calls made outside `Sequencer.plan` are not cached and always return fresh results. This fixes stale results outside planning. - Cached reports are defensively copied when stored and returned, so callers that mutate a report cannot alter the cached entry or other callers' reports. This fixes mutable report aliasing. Benchmark Workload: the real Snapcraft 9.1.3 `pack` planning path on the KF6 project (102 parts, 666 dependency edges, local SDK tree of about 11 GB and 152,115 files). Method: repeated randomized paired runs, 10 on tag 2.35.0 (baseline) and 10 on this branch (candidate). Each run targets `LifecycleManager.plan(PRIME)` and aborts before any lifecycle action executes, so only planning was measured. Build execution time was not measured. Results (means over 10 runs each): - Pack-to-plan elapsed time: 237.658 s -> 9.575 s. The mean paired reduction is 228.083 s (about 96.0%). - `LifecycleManager.plan` time: 233.789 s -> 5.197 s. - Repeated source scans: 419 -> 2. - Source-scan time: 230.134 s -> 1.288 s. All runs produced the same 408-action plan digest, so planning output is unchanged. The paired comparison showed 10/10 improvements. The 95% CI for the mean reduction is 224.066-232.101 s. The Holm-adjusted two-sided paired t-test gives p=1.60e-15, and the exact sign test gives p=0.001953.

Evaluation history

Date Model Scores Action Summary
~deepseek/deepseek-pro-latest
Impact: 75
Quick Win: 12
Complexity: 60
Confidence: 60
Actionability: 40
keep open Adds planning-scoped cache for source outdated checks in StateManager, keyed by part and pull-state serial, with defensive report copies. ~96% planning speedup on large SDK. CI failing lint/files and OSV-scanner; one unresolved review comment.

Update history

Date Change
created

Related issues

No related issues found above the similarity threshold.