Fixed by PR #2043 (commit 47fb62f3), which adds libffi-dev to the charm part's build-packages so cffi builds find ffi.h. Shipped in Charmcraft 3.3; maintainer closed the issue, marking the 2.7 path WONTFIX.
Resolution: closed resolved
Reason: Merged PR #2043 (commit 47fb62f3) added 'libffi-dev' to the charm plugin's deb build packages in charmcraft/parts/plugins/_charm.py, resolving the cffi/ffi.h build failure. Maintainer @lengau confirmed in a comment that it is solved in Charmcraft 3.3, and the 3.4 release notes list #2081 as fixed in 3.4.0.
Evaluated by:mr-cal-bot
No scores available.
Issue body
### Bug Description
`charmcraft` version `3.2.2` fail packing [`cos-proxy-operator`](https://github.com/canonical/cos-proxy-operator)
`charmcraft` version `2.7.3` pack [`cos-proxy-operator`](https://github.com/canonical/cos-proxy-operator) charm successfully.
Execution log: [charmcraft-20250116-193512.859515.log](https://github.com/user-attachments/files/18446098/charmcraft-20250116-193512.859515.log)
### To Reproduce
1. `git clone https://github.com/canonical/cos-proxy-operator`
2. `charmcraft clean`
3. `charmcraft pack --verbose --debug`
After that you'll see:
```shell
:: :: building '_cffi_backend' extension
:: :: creating build/temp.linux-x86_64-cpython-38/src/c
:: :: x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DFFI_BUILDING=1 -DUSE__THREAD -DHAVE_SYNC_SYNCHRONIZE -I/usr/include/ffi -I/usr/include/libffi -I/root/parts/charm/build/staging-venv/include -I/usr/include/python3.8 -c src/c/_cffi_backend.c -o build/temp.linux-x86_64-cpython-38/src/c/_cffi_backend.o
:: :: src/c/_cffi_backend.c:15:10: fatal error: ffi.h: No such file or directory
:: :: 15 | #include <ffi.h>
:: :: | ^~~~~~~
:: :: compilation terminated.
:: :: error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
:: :: [end of output]
:: ::
:: :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: :: ERROR: Failed building wheel for cffi
:: :: Failed to build cffi
:: :: ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (cffi)
:: ::
:: :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
:: :: [notice] To update, run: python3 -m pip install --upgrade pip
:: :: [end of output]
:: ::
:: :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: :: error: subprocess-exited-with-error
:: ::
:: :: × pip subprocess to install build dependencies did not run successfully.
:: :: │ exit code: 1
:: :: ╰─> See above for output.
:: ::
:: :: note: This error originates from a subprocess, and is likely not a problem with pip.
:: ::
:: :: [notice] A new release of pip is available: 24.1.1 -> 24.3.1
:: :: [notice] To update, run: python3 -m pip install --upgrade pip
:: Traceback (most recent call last):
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 483, in <module>
:: main()
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 478, in main
:: builder.build_charm()
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 93, in build_charm
:: self.handle_dependencies()
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 354, in handle_dependencies
:: self._install_dependencies(staging_venv_dir)
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/instrum.py", line 152, in _f
:: return func(*args, **kwargs)
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 284, in _install_dependencies
:: _process_run(
:: File "/snap/charmcraft/5303/lib/python3.10/site-packages/charmcraft/charm_builder.py", line 414, in _process_run
:: raise RuntimeError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
:: RuntimeError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip', 'install', '--no-binary=:all:', '--requirement=requirements.txt'] execution failed with retcode 1
Failed to run the build script for part 'charm'.
Launching shell on build environment...
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project#
```
In that shell you can verify that `libffi-dev` is not installed and the file `/usr/include/x86_64-linux-gnu/ffi.h` does not exist:
```shell
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# ls -l /usr/include/x86_64-linux-gnu/
total 52
-rw-r--r-- 1 root root 4351 Apr 14 2020 a.out.h
drwxr-xr-x 2 root root 4096 Nov 25 14:16 asm
drwxr-xr-x 3 root root 4096 Nov 25 14:16 bits
drwxr-xr-x 3 root root 4096 Nov 25 14:16 c++
-rw-r--r-- 1 root root 3830 Dec 2 00:26 expat_config.h
drwxr-xr-x 2 root root 4096 Nov 25 14:16 finclude
-rw-r--r-- 1 root root 3584 Apr 14 2020 fpu_control.h
drwxr-xr-x 2 root root 4096 Nov 25 14:16 gnu
-rw-r--r-- 1 root root 4916 Apr 14 2020 ieee754.h
drwxr-xr-x 2 root root 4096 Jan 15 10:22 python3.8
drwxr-xr-x 2 root root 4096 Nov 25 14:16 sys
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# dpkg -l | grep libffi-dev
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project#
```
But `libffi-dev` it is installable:
```shell
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# apt install libffi-dev
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
libffi-dev
0 upgraded, 1 newly installed, 0 to remove and 96 not upgraded.
Need to get 57.0 kB of archives.
After this operation, 312 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 libffi-dev amd64 3.3-4 [57.0 kB]
Fetched 57.0 kB in 1s (60.0 kB/s)
Selecting previously unselected package libffi-dev:amd64.
(Reading database ... 16342 files and directories currently installed.)
Preparing to unpack .../libffi-dev_3.3-4_amd64.deb ...
Unpacking libffi-dev:amd64 (3.3-4) ...
Setting up libffi-dev:amd64 (3.3-4) ...
```
After that installation the file `/usr/include/x86_64-linux-gnu/ffi.h` is there:
```shell
charmcraft-cos-proxy-on-amd64-for-amd64-62 ../project# ls -l /usr/include/x86_64-linux-gnu/ffi.h
-rw-r--r-- 1 root root 13934 Mar 23 2020 /usr/include/x86_64-linux-gnu/ffi.h
```
### Environment
- `charmcraft`: `3.2.2`
- `ubuntu`: `24.04.1 LTS`
### charmcraft.yaml
```yaml
type: charm
bases:
- build-on:
- name: "ubuntu"
channel: "20.04"
run-on:
- name: "ubuntu"
channel: "20.04"
- build-on:
- name: "ubuntu"
channel: "22.04"
run-on:
- name: "ubuntu"
channel: "22.04"
parts:
charm:
build-packages:
- "git"
# rustc and cargo required to build pydantic
- rustc
- cargo
charm-binary-python-packages:
# Install PyYAML from binary and avoid building it from sources. This way, we can use PyYAML with C-optimized lib.
# With the C-optimized lib, serialization in ops is 20x faster.
- PyYAML
- cryptography
nrpe-exporter:
plugin: dump
source: .
build-packages:
- curl
override-pull: |
if [ $CRAFT_ARCH_BUILD_FOR == "amd64" ]; then
URI=https://github.com/canonical/nrpe_exporter/releases/latest/download/nrpe_exporter-amd64
curl -L $URI -o nrpe_exporter-amd64
elif [ $CRAFT_ARCH_BUILD_FOR == "arm64" ] || [ $CRAFT_ARCH_BUILD_FOR == "aarch64" ]; then
URI=https://github.com/canonical/nrpe_exporter/releases/latest/download/nrpe_exporter-arm64
curl -L $URI -o nrpe_exporter-aarch64
fi
vector:
plugin: dump
source: .
build-packages:
- curl
- tar
override-pull: |
VERSION=0.44.0
if [ $CRAFT_ARCH_BUILD_FOR == "amd64" ]; then
ARCH=x86_64-unknown-linux-musl
elif [ $CRAFT_ARCH_BUILD_FOR == "arm64" ]; then
ARCH=aarch64-unknown-linux-musl
fi
VECTOR_URI="https://packages.timber.io/vector/${VERSION}/vector-${VERSION}-${ARCH}.tar.gz"
curl -sSfL ${VECTOR_URI} | tar xzf - -C . --strip-components=3 ./vector-${ARCH}/bin/vector
```
### Relevant log output
```shell
Execution log attached.
```
Evaluation history
Date
Model
Scores
Action
Summary
stealth/space-bunny-alpha
—
closed resolved
Fixed by PR #2043 (commit 47fb62f3), which adds libffi-dev to the charm part's build-packages so cffi builds find ffi.h. Shipped in Charmcraft 3.3; maintainer closed the issue, marking the 2.7 path WONTFIX.
stealth/space-bunny-alpha
—
closed resolved
Fixed in Charmcraft 3.3, promoted to stable on 2025-01-27. The missing libffi development dependency was added to the charm build environment; the Charmcraft 2.7 case was marked WONTFIX.
qwen3.6-35b-a3b-mtp-q6
—
—
Fixed in Charmcraft 3.3. The packaging failure caused by missing libffi-dev during cffi compilation was resolved. Legacy version 2.7 is marked WONTFIX; users should add libffi-dev to build-packages as a workaround.
qwen/qwen3.6-35b-a3b
—
—
Fixed in Charmcraft 3.3. The build environment now correctly includes dependencies required to compile cffi. The 2.7 series is marked WONTFIX; users can add libffi-dev to build-packages as a workaround.
qwen/qwen3.6-35b-a3b
—
—
Fixed in Charmcraft 3.3, resolving the missing libffi-dev dependency during packaging. The older 2.7 version was marked WONTFIX, with a workaround to manually add libffi-dev to build-packages.
qwen3.6-35b-a3b-mtp-q6
—
—
Resolved in Charmcraft 3.3. The missing libffi-dev dependency causing cffi compilation failures was fixed in the stable release. The 2.7.x branch is marked WONTFIX; users should add libffi-dev to build-packages as a workaround.
Merged PR 'fix: include libffi-dev in charm part build-packages', commit 47fb62f3; libffi-dev now listed in the charm plugin's get_build_packages for deb platforms.
cffi failed to build in charm builds because ffi.h was missing. Fixed by merged PR #2043 (commit 47fb62f3), which added libffi-dev to the charm part's build-packages.