Issues with concurrent builds
Metadata
Current evaluation
Concurrent charm builds triggered LXD race conditions and project conflicts. A fix resolving these concurrency issues is now included in Charmcraft, closing the issue.
Suggested action: —
No scores available.
Issue body
I am working on a test in [metallb-operator](https://github.com/charmed-kubernetes/metallb-operator) and was trying to build the two charms for that in parallel. It worked fine for me locally, but on the GitHub runner (a fresh VM, with a fresh install of both lxd and charmcraft on the default channel / track), I ran into a few errors that appear to be due to race conditions in setting up LXD for the builder.
The first one I [ran into](https://github.com/charmed-kubernetes/metallb-operator/pull/18/checks?check_run_id=3693265761#step:6:58) (NB: these build log links will eventually expire) was:
```
Packing charm 'metallb-speaker_ubuntu-20.04-amd64.charm'...
charmcraft internal error! LXDError: Failed to add remote 'craft-com.ubuntu.cloud-buildd'.
* Command that failed: 'lxc remote add craft-com.ubuntu.cloud-buildd https://cloud-images.ubuntu.com/buildd/releases --protocol=simplestreams'
* Command exit code: 1
* Command standard error output: b'Error: Remote craft-com.ubuntu.cloud-buildd exists as <https://cloud-images.ubuntu.com/buildd/releases>\n' (full execution logs in /home/runner/snap/charmcraft/common/charmcraft-log-osa2otki)
```
I tried working around that by running that `lxc` command manually earlier in the Workflow, but then I [ran into](https://github.com/charmed-kubernetes/metallb-operator/pull/18/checks?check_run_id=3700739113#step:7:58):
```
Packing charm 'metallb-controller_ubuntu-20.04-amd64.charm'...
charmcraft internal error! LXDError: Failed to create project 'charmcraft'.
* Command that failed: 'lxc --project charmcraft project create local:charmcraft'
* Command exit code: 1
* Command standard error output: b'Error: Failed creating project "charmcraft": Failed adding database record: This project already exists\n' (full execution logs in /home/runner/snap/charmcraft/common/charmcraft-log-4l5wgu0x)
```
Once again, I added the `lxc` command to the Workflow before the build, and then I [ran into](https://github.com/charmed-kubernetes/metallb-operator/pull/18/checks?check_run_id=3700929568#step:7:59):
```
Packing charm 'metallb-controller_ubuntu-20.04-amd64.charm'...
charmcraft internal error! LXDError: Failed to launch instance 'charmcraft-metallb-controller-1058053-0-0-amd64'.
* Command that failed: "lxc --project charmcraft launch craft-com.ubuntu.cloud-buildd:20.04 local:charmcraft-metallb-controller-1058053-0-0-amd64 --config 'raw.idmap=both 1001 0' --config security.syscalls.intercept.mknod=true"
* Command exit code: 1
* Command output: b'Creating charmcraft-metallb-controller-1058053-0-0-amd64\n'
* Command standard error output: b'Error: Failed instance creation: Failed creating instance record: Failed initialising instance: Invalid devices: Failed detecting root disk device: No root device could be found\n' (full execution logs in /home/runner/snap/charmcraft/common/charmcraft-log-jsos__q1)
```
That one has me stumped so I ended up just de-parallelizing the builds and that works, but it is slower and the errors were confusing and not immediately obvious that they were due to the concurrent build conflict.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen3.6-35b-a3b-mtp-q6 | — | — | Concurrent charm builds triggered LXD race conditions and project conflicts. A fix resolving these concurrency issues is now included in Charmcraft, closing the issue. | |
| qwen/qwen3.6-35b-a3b | — | — | Concurrent charm builds triggered LXD race conditions in charmcraft. The issue was fixed and the resolution was included in Charmcraft, eliminating the need for workarounds like de-parallelizing builds or using destructive mode. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Concurrent charm builds triggered LXD race conditions causing remote and project creation failures. Resolved by a fix included in Charmcraft. Workarounds included de-parallelizing builds or using destructive mode. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Race conditions in LXD caused failures during parallel charm builds. A fix resolving these concurrency conflicts is now included in Charmcraft. Users can also use destructive-mode as a workaround. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #391 LXD issue when build multiple charms in parallel | craft-providers | open | LXD race condition causing parallel charm build failures; reopened by maintainer after resurfacing in modern Charmcraft | |
| #1132 LXD race condition when packing multiple charms simultaneously | charmcraft | closed | Fixed in charmcraft and craft-providers. Parallel LXD commands like lxc project create and lxc copy caused race conditions during simultaneous charm packing. Concurrency controls resolved the issue in charmcraft edge and candidate 2.4.0. | |
| #506 Build a charmed failed | charmcraft | closed | charmcraft build failed due to an outdated LXD version lacking project support. Maintainers requested version details. Closed without resolution, likely abandoned, with a request to reopen if the problem persists. |