fix: migrate overlay files before running stage or prime
Metadata
Current evaluation
Merged #1617 as 2d81a4bd, moving overlay migration before stage/prime so build-created directories cannot shadow overlay state and break priming; regression coverage was added.
Resolution: closed resolved
Reason: PR #1617 was approved and merged with all CI checks passing. Commit 2d81a4bd reorders overlay migration ahead of stage or prime execution and adds a regression test for the failing scenario.
Evaluated by: mr-cal-bot
No scores available.
Issue body
This commit switches the order of calling _run_step() and _migrate_overlay* so that the overlay migration always happens *before* the default step behavior. This is the desired behavior, but the previous code had a corner-case when a part has content coming from both 'build' and 'overlay', like this:
```yaml
parts:
p1:
override-build: |
mkdir ${CRAFT_PART_INSTALL}/my-dir
override-overlay: |
mkdir my-dir
touch my-dir/my-file
```
... in this situation, the previous code would *first* stage the `my-dir`directory coming from `override-build` and *then* migrate `my-dir` and `my-dir/my-file` from the overlay to stage. Because of the way the migration happens, the `my-dir` dir coming from build would "shadow" the `my-dir` dir coming from stage, which meant the migrate state would *not* record `my-dir` as coming from the overlay. This is a bug in and of itself but it was usually harmless.
A failure happens if another, separate part gets primed *before* `p1`. In that case, we would try to do the migration of overlay contents from stage to prime, which in this case meant migrating the `my-dir/my-file` file but **not** the `my-dir` dir because it did not get recorded. Then, the migration of `my-dir/my-file` would *maybe* work:
- if `my-file` is a regular file, the code would follow a path that recreates the directory structure to hardlink the file (to save storage);
- if `my-file` is a special file like a symlink the code would try to copy it to `${prime}/my-dir/`, which doesn't exist. Then a failure happens with a cryptic message that suggests that `${stage}/my-dir/my-file` doesn't exist, but this is just bad formatting of the error message.
In my view the main cause of this bug is the fact that overlay files are not correctly recorded in the state file - I added a test to check this scenario and I think it's sufficient to validate this particular bugfix.
---
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Merged #1617 as 2d81a4bd, moving overlay migration before stage/prime so build-created directories cannot shadow overlay state and break priming; regression coverage was added. | |
| stealth/space-bunny-alpha | — | closed resolved | Merged in canonical/craft-parts#1617 (2d81a4bd): overlay files now migrate before stage or prime, preventing build-created directories from shadowing overlay state and breaking priming. | |
| qwen/qwen3.6-35b-a3b | — | — | Merged fix that migrates overlay files before stage or prime steps. Resolves a state tracking bug causing priming failures with symlinks. Approved by reviewers and passed all CI checks. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Fixed overlay migration order to precede stage and prime steps, resolving a bug where overlay contents were shadowed by build artifacts and caused prime failures. Merged with a validation test. |
Update history
No update history recorded yet.
Related work
-
Related To:
canonical/craft-parts#1619
(confidence 85%)
Commit bf3615ca references #1619 and carries the same #1617 fix, indicating a backport or forward-port.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1619 fix: migrate overlay files before running stage or prime (#1617) | craft-parts | merged | Merged as bf3615ca, cherry-picking #1617 so overlay files migrate before stage or prime, preventing build-created directories from shadowing overlay state and breaking priming. | |
| #121 overlay: migrate overlay data to stage and prime (CRAFT-468, CRAFT-82) | craft-parts | merged | Merged PR #121 as d45d7079, moving overlay data into stage/prime, translating whiteouts and opaque directories to OCI, and cleaning shared overlay data. | |
| #1123 fix: Prime content from overlay when split into partitions | craft-parts | merged | Merged PR #1123 (commit 3540e306), fixing overlay priming with partitions by using stage content and migration state while preserving existing prime state files. | |
| #1190 fix(overlay): detect conflicts between overlay and install | craft-parts | merged | Merged PR #1190 fixed Stage collision detection by including overlay layer contents and added integration tests for colliding and non-colliding scenarios in commit b7455277. | |
| #1618 fix: handle dependencies when organizing to overlay | craft-parts | merged | Merged PR #1618 (commit 18239b07) fixes dependency scheduling when organizing to overlay, prioritizing BUILD/organize over dependency STAGE steps and closing #1611. | |
| #1140 Overlay contents don't conflict with regular stage files | craft-parts | closed | Overlay/install collisions are now detected. PR #1190 was merged as commit b7455277, adding overlay contents to stage conflict checks and integration tests. |