← Back to issue list

Issues with concurrent builds

View original Github issue

Metadata

Project
charmcraft
Number
#554
Type
issue
State
closed
Author
johnsca
Labels
Created
Updated
Closed

Current evaluation

Concurrent charm builds triggered LXD race conditions and project conflicts. A fix resolving these concurrency issues is now included in Charmcraft, closing the issue.

Suggested action:

No scores available.

Issue body

I am working on a test in [metallb-operator](https://github.com/charmed-kubernetes/metallb-operator) and was trying to build the two charms for that in parallel. It worked fine for me locally, but on the GitHub runner (a fresh VM, with a fresh install of both lxd and charmcraft on the default channel / track), I ran into a few errors that appear to be due to race conditions in setting up LXD for the builder. The first one I [ran into](https://github.com/charmed-kubernetes/metallb-operator/pull/18/checks?check_run_id=3693265761#step:6:58) (NB: these build log links will eventually expire) was: ``` Packing charm 'metallb-speaker_ubuntu-20.04-amd64.charm'... charmcraft internal error! LXDError: Failed to add remote 'craft-com.ubuntu.cloud-buildd'. * Command that failed: 'lxc remote add craft-com.ubuntu.cloud-buildd https://cloud-images.ubuntu.com/buildd/releases --protocol=simplestreams' * Command exit code: 1 * Command standard error output: b'Error: Remote craft-com.ubuntu.cloud-buildd exists as <https://cloud-images.ubuntu.com/buildd/releases>\n' (full execution logs in /home/runner/snap/charmcraft/common/charmcraft-log-osa2otki) ``` I tried working around that by running that `lxc` command manually earlier in the Workflow, but then I [ran into](https://github.com/charmed-kubernetes/metallb-operator/pull/18/checks?check_run_id=3700739113#step:7:58): ``` Packing charm 'metallb-controller_ubuntu-20.04-amd64.charm'... charmcraft internal error! LXDError: Failed to create project 'charmcraft'. * Command that failed: 'lxc --project charmcraft project create local:charmcraft' * Command exit code: 1 * Command standard error output: b'Error: Failed creating project "charmcraft": Failed adding database record: This project already exists\n' (full execution logs in /home/runner/snap/charmcraft/common/charmcraft-log-4l5wgu0x) ``` Once again, I added the `lxc` command to the Workflow before the build, and then I [ran into](https://github.com/charmed-kubernetes/metallb-operator/pull/18/checks?check_run_id=3700929568#step:7:59): ``` Packing charm 'metallb-controller_ubuntu-20.04-amd64.charm'... charmcraft internal error! LXDError: Failed to launch instance 'charmcraft-metallb-controller-1058053-0-0-amd64'. * Command that failed: "lxc --project charmcraft launch craft-com.ubuntu.cloud-buildd:20.04 local:charmcraft-metallb-controller-1058053-0-0-amd64 --config 'raw.idmap=both 1001 0' --config security.syscalls.intercept.mknod=true" * Command exit code: 1 * Command output: b'Creating charmcraft-metallb-controller-1058053-0-0-amd64\n' * Command standard error output: b'Error: Failed instance creation: Failed creating instance record: Failed initialising instance: Invalid devices: Failed detecting root disk device: No root device could be found\n' (full execution logs in /home/runner/snap/charmcraft/common/charmcraft-log-jsos__q1) ``` That one has me stumped so I ended up just de-parallelizing the builds and that works, but it is slower and the errors were confusing and not immediately obvious that they were due to the concurrent build conflict.

Evaluation history

Date Model Scores Action Summary
qwen3.6-35b-a3b-mtp-q6 Concurrent charm builds triggered LXD race conditions and project conflicts. A fix resolving these concurrency issues is now included in Charmcraft, closing the issue.
qwen/qwen3.6-35b-a3b Concurrent charm builds triggered LXD race conditions in charmcraft. The issue was fixed and the resolution was included in Charmcraft, eliminating the need for workarounds like de-parallelizing builds or using destructive mode.
qwen3.6-35b-a3b-mtp-q6 Concurrent charm builds triggered LXD race conditions causing remote and project creation failures. Resolved by a fix included in Charmcraft. Workarounds included de-parallelizing builds or using destructive mode.
qwen3.6-35b-a3b-mtp-q6 Race conditions in LXD caused failures during parallel charm builds. A fix resolving these concurrency conflicts is now included in Charmcraft. Users can also use destructive-mode as a workaround.

Update history

No update history recorded yet.

Related issues

Issue Project State Summary Similarity
#391 LXD issue when build multiple charms in parallel craft-providers open LXD race condition causing parallel charm build failures; reopened by maintainer after resurfacing in modern Charmcraft
84%
#1132 LXD race condition when packing multiple charms simultaneously charmcraft closed Fixed in charmcraft and craft-providers. Parallel LXD commands like lxc project create and lxc copy caused race conditions during simultaneous charm packing. Concurrency controls resolved the issue in charmcraft edge and candidate 2.4.0.
81%
#506 Build a charmed failed charmcraft closed charmcraft build failed due to an outdated LXD version lacking project support. Maintainers requested version details. Closed without resolution, likely abandoned, with a request to reopen if the problem persists.
71%