← Back to issue list

`build-on` when running snapcraft locally uses stage packages from the wrong architecture

View original Github issue

Metadata

Project
snapcraft
Number
#5335
Type
issue
State
closed
Author
olivercalder
Labels
Status: Triaged Type: Question
Created
Updated
Closed

Current evaluation

Closed as expected behavior. Snapcraft uses the build-on architecture for stage-packages. Cross-compilation requires suffixing packages with :$CRAFT_ARCH_BUILD_FOR. Correct usage is now documented in the official Snapcraft guides.

Suggested action:

No scores available.

Issue body

### Bug Description I'm experimenting with building a simple snap of mine on all available architectures. The snap is based on `core24`, and depends on the `qrencode` package. The full snap can be found at https://github.com/olivercalder/qr-server. When I ran snapcraft after adding a `build-on: [amd64, ...]` directive for each available architecture, I noticed something strange: ``` snapcraft Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_amd64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_arm64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_armhf.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_ppc64el.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_riscv64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_s390x.snap ``` No matter which architecture is being built, there's an error about downloading `libqrencode4_4.1.1-1build2_amd64.deb`. I'd expect this file to match the name of the architecture specified. But snapcraft happily built artifacts for each architecture, and the `meta/snap.yaml` inside each one reported its correct architecture. To be sure, I copied the `arm64` build to a rpi4 and installed it. It installed just fine, but when I made a request to the flask API (which invokes `qrencode` directly --- it's a very dumb and simple application), it errors with this error ``` Mar 16 17:15:48 pi qr-server.qr-server[43836]: OSError: [Errno 8] Exec format error: 'qrencode' ``` Just to check, I ran `file` on the `qrencode` binary in the snap, and sure enough: ``` me@rpi4:~$ file /snap/qr-server/x1/usr/bin/qrencode /snap/qr-server/x1/usr/bin/qrencode: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=a778f62a63750bc9732021db04d8eb3fe4202b0d, for GNU/Linux 3.2.0, stripped ``` ---- It seems there are one of two bugs: - `snapcraft` is incorrectly attempting to cross-build snaps and reporting that it succeeded - From my reading of the snapcraft reference documentation, I thought cross-building could only be done using a remote build provider (launchpad), so I expected it to only build the native architecture of the snap - `snapcraft` is incorrectly pulling the native package dependencies from `stage-packages` - The `python3` binary and other implicit packages seemed to run just fine, but as soon as python invoked `qrencode`, which was a staged package, it failed because the binary was `x86_64` instead of `aarch64`. ### To Reproduce Here's how to replicate exactly what I tried. I'm running snapcraft 8.7.2 from `latest/stable` with lxd backend on Ubuntu 24.10, assuming your primary machine is amd64. ```bash # Clone my snap repo git clone https://github.com/olivercalder/qr-server -b v1.0.0 cd qr-server # Apply the patch which adds `build-on` directives so every arch can be built on amd64 git apply - <<EOF diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml index e029242..572c3a5 100644 --- a/snap/snapcraft.yaml +++ b/snap/snapcraft.yaml @@ -31,11 +31,17 @@ issues: https://github.com/olivercalder/qr-server/issues platforms: amd64: + build-on: [amd64, arm64] arm64: + build-on: [amd64, arm64] armhf: + build-on: [amd64, arm64, armhf] ppc64el: + build-on: [amd64, arm64, ppc64el] riscv64: + build-on: [amd64, arm64, riscv64] s390x: + build-on: [amd64, arm64, s390x] grade: stable confinement: strict EOF # Invoke snapcraft (pay attention to snapcraft # Check that a snap was built for all architectures ls qr-server_1.0.0_{amd64,arm64,armhf,ppc64el,riscv64,s390x}.snap # Check the architecture of the qrencode library bundled in each of the snap files for arch in amd64 arm64 armhf ppc64el riscv64 s390x; do unsquashfs "qr-server_1.0.0_$arch.snap" echo "Inspecting qrencode binary from within snap built for $arch" file squashfs-root/usr/bin/qrencode rm -rf squashfs-root done ``` ### Environment I am running snapcraft 8.7.2 (13860) from latest/stable on Ubuntu 24.10, using lxd as a backend. ### snapcraft.yaml ```yaml name: qr-server base: core24 version: '1.0.0' summary: A server to generate QR codes description: | Listen on a port and serve QR codes encoding data from the path or the request body. For example, if running on localhost and port 5000, then both of the following are supported ways to generate a QR code encoding the string "hello-there": - `curl http://localhost:5000/hello-there` - `curl --data-binary 'hello-there' http://localhost:5000` If the path following the hostname is non-empty, that will be used as the data to encode, and the request body will be ignored. By default, listens on port 5000. To listen on another port, use `sudo snap set qr-server port=<port>`, with `<port>` being any available port number. Both GET and POST requests are accepted and may be used interchangeably. title: QR Server icon: icon.png license: MIT source-code: https://github.com/olivercalder/qr-server website: https://github.com/olivercalder/qr-server issues: https://github.com/olivercalder/qr-server/issues platforms: amd64: build-on: [amd64, arm64] arm64: build-on: [amd64, arm64] armhf: build-on: [amd64, arm64, armhf] ppc64el: build-on: [amd64, arm64, ppc64el] riscv64: build-on: [amd64, arm64, riscv64] s390x: build-on: [amd64, arm64, s390x] grade: stable confinement: strict parts: qr-server: plugin: dump source: bin/ organize: qr-server: bin/qr-server qr_server.py: bin/qr_server.py stage-packages: - qrencode python-dependencies: plugin: python source: . python-packages: - flask apps: qr-server: command: bin/qr-server daemon: simple plugs: - network-bind ``` ### Relevant log output ```shell Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_amd64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_arm64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_armhf.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_ppc64el.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_riscv64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_s390x.snap ``` ### Additional context _No response_

Evaluation history

Date Model Scores Action Summary
qwen/qwen3.6-35b-a3b Closed as expected behavior. Snapcraft uses the build-on architecture for stage-packages. Cross-compilation requires suffixing packages with :$CRAFT_ARCH_BUILD_FOR. Correct usage is now documented in the official Snapcraft guides.
qwen/qwen3.6-35b-a3b Closed as expected behavior. Maintainer clarified local cross-compilation requires suffixing stage packages with :$CRAFT_ARCH_BUILD_FOR and adding the ubuntu ports repository. Documentation gap addressed with official guides.
qwen3.6-35b-a3b-mtp-q6 Closed as expected behavior. Maintainers clarified that local cross-compilation requires suffixing stage packages with :$CRAFT_ARCH_BUILD_FOR and adding the ubuntu ports repository. The correct workflow is now documented in the official Snapcraft docs.
qwen3.6-35b-a3b-mtp-q6 Resolved as expected behavior. Local builds with build-on use the host architecture for stage-packages. Cross-compilation requires suffixing packages with :$CRAFT_ARCH_BUILD_FOR and adding ubuntu ports. The workflow is now documented in official Snapcraft guides.

Update history

No update history recorded yet.

Related issues

Issue Project State Summary Similarity
#2012503 wrong arch of stage packages snapcraft (launchpad) open Snapcraft 7.3 picks host architecture for stage packages when cross-building (build-on amd64, build-for arm64). No maintainer response in 3.4 years. Nearly identical issue #5335 was closed as expected behavior requiring explicit :$CRAFT_ARCH_BUILD_FOR suffix on stage packages.
75%
#1885150 Cannot use --build-on, architecture list is already set in snapcraft.yaml. snapcraft (launchpad) closed Closed without fix due to inactivity. The --build-on CLI flag remains ignored when architectures is defined in snapcraft.yaml, requiring manual YAML edits for architecture-specific builds.
73%
#1921418 wrong architecture of stage-snaps when cross building snapcraft (launchpad) open Cross-building with stage-snaps downloads the wrong architecture snap (tested on snapcraft 4.6). No comments, no activity for ~5 years, tracked under Jira label craft-415.
72%
#4150 meta: use build-for in snap.yaml architecture snapcraft merged Merged fix correcting snap.yaml architecture generation. When build-for is set to all, the file now correctly outputs architectures: all instead of the previous incorrect build-on value. Resolves CRAFT-1535.
71%