Race condition when mounting loop devices
Metadata
Current evaluation
Race condition where loop partition devices (/dev/loopXpN) don't exist yet when mount is called after losetup --partscan. Triaged, maintainer suggested flock on parent device. Partial fix (wait for nodes) added in services/image.py via PR #428, but same failure still hit in pack path CI (19 days ago).
Suggested action: keep open
Reason: Issue is triaged and acknowledged by maintainer mwhudson who confirmed the race and suggested flock on the parent device. A partial fix (_wait_for_partition_nodes) was added in services/image.py (PR #428), but the pack/image.py path still lacks any wait, and the same class of failure was hit 19 days ago on PR #402 (non-sequential-partitions test). The issue remains reproducible and unfixed in the pack code path.
Impact:
55
Quick Win:
19.25
Staleness:
25
Complexity:
65
Confidence:
72
Support Request:
5
Issue body
### Check existing issues
- [x] I've verified that this bug isn't described by any existing issues.
### Bug description
Discovered and researched by @smethnani.
When we create loop devices, there's a race condition we sometimes hit when trying to mount them before they fully exist.
The upstream version of this bug is in https://bugs.launchpad.net/ubuntu/+source/util-linux/+bug/2045586
### Steps to reproduce
n/a it's a race condition
### Environment
We need to understand the context in which Imagecraft failed. Please let us know if you are running Imagecraft in destructive mode, whether you are using LXD or Multipass, and which operating system you are running Imagecraft on.
### imagecraft.yaml
```yaml
```
### Log output
```shell
See: https://github.com/canonical/imagecraft/actions/runs/23426535878/job/68143721209?pr=289#step:9:196
Getting partition table
Adding partition volume/pc/data3 to the image
Setting up GRUB in the image
mount: /home/imagecraft/tests/spread/pack/complex/mount: special device /dev/loop7p2 does not exist.
dmesg(1) may have more information after failed mount system call.
mount: /home/imagecraft/tests/spread/pack/complex/mount: mount failed: Unknown error 5005.
Failed to clean chroot
Detailed information: Command '['/bin/mount', '/home/imagecraft/tests/spread/pack/complex/mount', '--make-rprivate']' returned non-zero exit status 32.
Full execution log: '/root/.local/state/imagecraft/log/imagecraft-20260323-081704.190556.log'
-----
.
2026-03-23 08:17:22 Debug output for google:ubuntu-24.04-64:tests/spread/pack/complex (mar230753-821890) :
-----
++ find /root/.local/state/imagecraft/log/ -name 'imagecraft*.log'
++ sort -n
++ tail -n1
+ imagecraft_log_file=/root/.local/state/imagecraft/log/imagecraft-20260323-081704.190556.log
[...]
2026-03-23 08:17:21.218 Getting partition table
2026-03-23 08:17:21.218 Running command: ['sfdisk', '--json', 'pc.img']
2026-03-23 08:17:21.222 Adding partition volume/pc/data3 to the image
2026-03-23 08:17:21.286 Running command: ['sfdisk', '--json', 'pc.img']
2026-03-23 08:17:21.289 Setting up GRUB in the image
2026-03-23 08:17:21.289 Running command: ['losetup', '--find', '--show', '--partscan', 'pc.img']
2026-03-23 08:17:21.313 Attached image pc.img as loop device /dev/loop7
2026-03-23 08:17:21.314 [pid=57966] parent process
2026-03-23 08:17:21.314 [pid=57966] set up chroot
2026-03-23 08:17:21.314 setup chroot: PosixPath('/home/imagecraft/tests/spread/pack/complex/mount')
2026-03-23 08:17:21.314 [pid=57966] mount '/home/imagecraft/tests/spread/pack/complex/mount' on chroot
2026-03-23 08:17:21.315 mount device='/dev/loop7p2', mountpoint='/home/imagecraft/tests/spread/pack/complex/mount', args=()
2026-03-23 08:17:21.318 [pid=57966] clean up chroot
2026-03-23 08:17:21.318 cleanup chroot: PosixPath('/home/imagecraft/tests/spread/pack/complex/mount')
2026-03-23 08:17:21.318 [pid=57966] umount: '/home/imagecraft/tests/spread/pack/complex/mount'
2026-03-23 08:17:21.318 mount device='/home/imagecraft/tests/spread/pack/complex/mount', mountpoint='--make-rprivate', args=()
2026-03-23 08:17:21.321 Running command: ['losetup', '--json']
2026-03-23 08:17:21.325 Detaching loop device /dev/loop7 (from /home/imagecraft/tests/spread/pack/complex/pc.img)
2026-03-23 08:17:21.325 Running command: ['losetup', '-d', '/dev/loop7']
2026-03-23 08:17:21.327 Failed to clean chroot
2026-03-23 08:17:21.327 Detailed information: Command '['/bin/mount', '/home/imagecraft/tests/spread/pack/complex/mount', '--make-rprivate']' returned non-zero exit status 32.
2026-03-23 08:17:21.328 Full execution log: '/root/.local/state/imagecraft/log/imagecraft-20260323-081704.190556.log'
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.8-27b |
Impact:
55
Quick Win:
19.25
Staleness:
25
Complexity:
65
Confidence:
72
Support Request:
5
|
keep open | Race condition where loop partition devices (/dev/loopXpN) don't exist yet when mount is called after losetup --partscan. Triaged, maintainer suggested flock on parent device. Partial fix (wait for nodes) added in services/image.py via PR #428, but same failure still hit in pack path CI (19 days ago). | |
| qwen/qwen3.6-35b-a3b |
Staleness:
10
Complexity:
45
Confidence:
90
Support Request:
5
|
keep open | Intermittent race condition causes loop device partitions to disappear before mounting, leading to CI test failures. Triaged and acknowledged by maintainers, with a proposed workaround but no fix yet. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
45
Complexity:
65
Confidence:
85
Support Request:
5
|
keep open | Race condition causing mount failures on loop devices due to udev processing delays. Triaged bug with maintainer suggesting flock-based synchronization. Awaiting implementation or further investigation. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
45
Complexity:
45
Confidence:
90
Support Request:
5
|
keep open | Race condition causes mount failures when loop partition devices disappear before udevd finishes processing. Triaged bug with a maintainer-proposed fix, awaiting implementation. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
45
Complexity:
65
Confidence:
85
Support Request:
5
|
keep open | Race condition during loop device mounting causes failures when partitions disappear before udevd finishes processing. Triaged, with a maintainer suggesting flock-based synchronization. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
45
Complexity:
30
Confidence:
90
Support Request:
0
|
keep open | Loop device partition mounts fail due to a race condition before udev finishes processing. Triaged bug. Suggested fix uses flock on the parent device during mount. Awaiting implementation. |
Update history
| Date | Change |
|---|---|
| updated |
Related work
-
Related To:
imagecraft#428
(confidence 80%)
PR #428 added _wait_for_partition_nodes in services/image.py which partially addresses the race by polling for partition node existence after losetup --partscan, but does not cover the pack/image.py path or the udev remove/recreate race mwhudson described.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #340 fix: Avoid races with udev and loop devices | imagecraft | open | Adds flock LOCK_EX on the whole-disk node during mkfs and across loop partition mounts to fix udev/loop-device races; supersedes #330's approach. Open 118 days with no review or comments; CI mostly green but lint, title policy, and snap-builds checks failing. |