Charmcraft hangs or fails with LXD 5.18
Metadata
Current evaluation
Closed due to an upstream LXD bug. A fix was released for LXD 5.18 compatibility, resolving the hangs and failures. The investigation also uncovered a separate craft-providers issue tracked independently.
Suggested action: —
No scores available.
Issue body
### Bug Description
Charmcraft has been hanging with latest/edge lxd.
```
$ sudo snap info lxd
...
tracking: latest/edge <----------------------------
refresh-date: today at 15:28 CEST <----------------------------------
channels:
latest/stable: 5.17-e5ead86 2023-08-30 (25505) 184MB -
latest/candidate: 5.17-e5ead86 2023-08-25 (25505) 184MB -
latest/beta: ↑
latest/edge: git-87fbf8a 2023-09-18 (25653) 189MB -
5.17/stable: 5.17-e5ead86 2023-08-29 (25505) 184MB -
```
Running with charmcraft in latest/stable fails with:
```
$ charmcraft pack --debug -v
Starting charmcraft version 2.4.1
Packing the charm.
Launching environment to pack for base name='ubuntu' channel='22.04' architectures=['amd64'] (may take a while the first time but it's reusable)
Checking for instance 'charmcraft-mongodb-k8s-12244861-0-0-amd64' in project 'charmcraft' in remote 'local'
Instance 'charmcraft-mongodb-k8s-12244861-0-0-amd64' does not exist.
Checking for base instance 'base-instance-charmcraft-buildd-base-v10-689fe40991089812a57b' in project 'charmcraft' in remote 'local'
Base instance 'base-instance-charmcraft-buildd-base-v10-689fe40991089812a57b' does not exist.
Creating new instance from image 'core22' from remote 'craft-com.ubuntu.cloud-buildd'.
charmcraft internal error: LXDError(brief="Failed to launch instance 'charmcraft-mongodb-k8s-12244861-0-0-amd64'.", details='* Command that failed: \'lxc --project charmcraft launch craft-com.ubuntu.cloud-buildd:core22 local:charmcraft-mongodb-k8s-12244861-0-0-amd64 --config security.syscalls.intercept.mknod=true\'\n* Command exit code: 1\n* Command output: b\'Creating charmcraft-mongodb-k8s-12244861-0-0-amd64\\n\'\n* Command standard error output: b"Error: Failed instance creation: Failed getting remote image info: Failed getting image: The requested image couldn\'t be found\\n"', resolution=None)
```
Refreshing charmcraft to latest edge and it hangs on:
```
...
2023-09-18 16:26:42.128 Command '['lxc', '--project', 'charmcraft', 'launch', 'craft-com.ubuntu.cloud-buildd:core22', 'local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50', '--config', 'security.syscalls.intercept.mknod=true', '--config', 'user.craft_providers.status=STARTING', '--config', 'user.craft_providers.timer=2023-09-18T14:26:41.949784+00:00']' returned non-zero exit status 1.
2023-09-18 16:26:42.128 Failed to launch instance base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50, checking status.
2023-09-18 16:26:42.128 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:45.195 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:48.287 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:51.367 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:54.455 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:57.535 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
...
## STAYS UNTIL TIMEOUT
```
### To Reproduce
```
# Delete every image in lxc's charmcraft project, including base image
$ sudo lxc delete --force ...
$ git clone https://github.com/canonical/mongodb-operator
# Refresh lxd
$ sudo snap refresh lxd --channel=latest/edge
# Checkout charmcraft
$ sudo snap install charmcraft
# Now build
$ charmcraft pack --debug -v
# Fails, try with edge
$ sudo snap refresh charmcraft --channel=latest/edge
$ charmcraft pack --debug -v # hangs
# Try removing ~/.local/state/charmcraft
# Same results
### Environment
Ubuntu 22.04
Using LXD
### charmcraft.yaml
```shell
https://github.com/canonical/mongodb-operator/blob/d9e79262dbc8a57bd7de39c817f526a0914b4adc/charmcraft.yaml#L4
```
### Relevant log output
```shell
...
2023-09-18 16:26:42.128 Command '['lxc', '--project', 'charmcraft', 'launch', 'craft-com.ubuntu.cloud-buildd:core22', 'local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50', '--config', 'security.syscalls.intercept.mknod=true', '--config', 'user.craft_providers.status=STARTING', '--config', 'user.craft_providers.timer=2023-09-18T14:26:41.949784+00:00']' returned non-zero exit status 1.
2023-09-18 16:26:42.128 Failed to launch instance base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50, checking status.
2023-09-18 16:26:42.128 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:45.195 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:48.287 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:51.367 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:54.455 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
2023-09-18 16:26:57.535 Executing on host: lxc --project charmcraft info local:base-instance-charmcraft-buildd-base-v20-2046f14fb4a00aa9ce50
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed due to an upstream LXD bug. A fix was released for LXD 5.18 compatibility, resolving the hangs and failures. The investigation also uncovered a separate craft-providers issue tracked independently. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed as an upstream LXD bug. Reverting to stable LXD resolves the hang. Upstream released a fix, but the report also exposed a separate craft-providers issue. Linked for reference. | |
| qwen/qwen3.6-35b-a3b | — | — | Upstream LXD released a fix for the instance launch failure causing charmcraft to hang or fail with LXD 5.18. This issue is closed as resolved by the upstream patch, with a related craft-providers bug linked for reference. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed. Upstream LXD bug #12295 was fixed, resolving the hang. Rolling back to stable LXD served as a temporary workaround. The report also exposed a secondary craft-providers issue. Closed for reference. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Superseded by an upstream LXD fix. The issue is closed, with LXD stable serving as a temporary workaround. It also uncovered a separate craft-providers bug tracked independently. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1640 Incompatibility with latest LXD LTS | charmcraft | closed | charmcraft failed parsing LXD 5.21 LTS version strings due to the appended LTS suffix. The issue was resolved by upgrading craft-providers to 1.23.1 and releasing the fix in charmcraft 2.6.0, now available on stable. | |
| #1047 Charmcraft stuck on "launching environment" | charmcraft | closed | Resolved by craft-providers 1.8, which now enforces timeouts and verifies network connectivity during LXD environment setup. This prevents indefinite hangs from networking or iptables conflicts, replacing silent failures with actionable error messages. | |
| #511 `charmcraft pack` fails running `apt update` | charmcraft | closed | Closed as out of scope. The failure stemmed from an LXD container networking bug, not charmcraft. No code changes were implemented; users were directed to external LXD and Discourse discussions for troubleshooting. | |
| #560 Charmcraft pack is giving error frequently | charmcraft | closed | Charmcraft pack failed due to an apt-get network error installing snapd in an LXD container. Maintainers linked it to a known networking issue. The reporter confirmed a workaround failed. Closed without a code fix, likely abandoned. | |
| #506 Build a charmed failed | charmcraft | closed | charmcraft build failed due to an outdated LXD version lacking project support. Maintainers requested version details. Closed without resolution, likely abandoned, with a request to reopen if the problem persists. | |
| #647 `charmcraft build` sometimes fails on `apt install` inside the lxd container | charmcraft | closed | Closed as a duplicate of issue #567. The report details intermittent apt-get lock failures during charmcraft build inside LXD containers, suspected to be caused by unattended-upgrades. |