← Back to issue list

`build-on` when running snapcraft locally uses stage packages from the wrong architecture

View original Github issue

Metadata

Project
snapcraft
Number
#5335
Type
issue
State
closed
Author
olivercalder
Labels
Status: Triaged Type: Question
Created
Updated
Closed

Current evaluation

Closed as expected behavior, not a bug: mr-cal explained build-on uses build-on arch for build/stage-packages; cross-compiling requires :$CRAFT_ARCH_BUILD_FOR suffixes plus ubuntu-ports repo. Later documented in craft-a-cross-compiled-app guide.

Resolution: closed not a bug

Reason: Maintainer mr-cal stated the behavior is by design — snapcraft resolves stage/build-packages for the build-on architecture, so cross-compiling requires explicit :$CRAFT_ARCH_BUILD_FOR package suffixes and added package repositories (tracked as internal CRAFT-4355). The gap was documentation only, addressed by the cross-compiled app how-to guide now present at docs/how-to/integrations/craft-a-cross-compiled-app.rst (added in commit 47574a240).

Evaluated by: mr-cal-bot

No scores available.

Issue body

### Bug Description I'm experimenting with building a simple snap of mine on all available architectures. The snap is based on `core24`, and depends on the `qrencode` package. The full snap can be found at https://github.com/olivercalder/qr-server. When I ran snapcraft after adding a `build-on: [amd64, ...]` directive for each available architecture, I noticed something strange: ``` snapcraft Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_amd64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_arm64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_armhf.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_ppc64el.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_riscv64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_s390x.snap ``` No matter which architecture is being built, there's an error about downloading `libqrencode4_4.1.1-1build2_amd64.deb`. I'd expect this file to match the name of the architecture specified. But snapcraft happily built artifacts for each architecture, and the `meta/snap.yaml` inside each one reported its correct architecture. To be sure, I copied the `arm64` build to a rpi4 and installed it. It installed just fine, but when I made a request to the flask API (which invokes `qrencode` directly --- it's a very dumb and simple application), it errors with this error ``` Mar 16 17:15:48 pi qr-server.qr-server[43836]: OSError: [Errno 8] Exec format error: 'qrencode' ``` Just to check, I ran `file` on the `qrencode` binary in the snap, and sure enough: ``` me@rpi4:~$ file /snap/qr-server/x1/usr/bin/qrencode /snap/qr-server/x1/usr/bin/qrencode: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, BuildID[sha1]=a778f62a63750bc9732021db04d8eb3fe4202b0d, for GNU/Linux 3.2.0, stripped ``` ---- It seems there are one of two bugs: - `snapcraft` is incorrectly attempting to cross-build snaps and reporting that it succeeded - From my reading of the snapcraft reference documentation, I thought cross-building could only be done using a remote build provider (launchpad), so I expected it to only build the native architecture of the snap - `snapcraft` is incorrectly pulling the native package dependencies from `stage-packages` - The `python3` binary and other implicit packages seemed to run just fine, but as soon as python invoked `qrencode`, which was a staged package, it failed because the binary was `x86_64` instead of `aarch64`. ### To Reproduce Here's how to replicate exactly what I tried. I'm running snapcraft 8.7.2 from `latest/stable` with lxd backend on Ubuntu 24.10, assuming your primary machine is amd64. ```bash # Clone my snap repo git clone https://github.com/olivercalder/qr-server -b v1.0.0 cd qr-server # Apply the patch which adds `build-on` directives so every arch can be built on amd64 git apply - <<EOF diff --git a/snap/snapcraft.yaml b/snap/snapcraft.yaml index e029242..572c3a5 100644 --- a/snap/snapcraft.yaml +++ b/snap/snapcraft.yaml @@ -31,11 +31,17 @@ issues: https://github.com/olivercalder/qr-server/issues platforms: amd64: + build-on: [amd64, arm64] arm64: + build-on: [amd64, arm64] armhf: + build-on: [amd64, arm64, armhf] ppc64el: + build-on: [amd64, arm64, ppc64el] riscv64: + build-on: [amd64, arm64, riscv64] s390x: + build-on: [amd64, arm64, s390x] grade: stable confinement: strict EOF # Invoke snapcraft (pay attention to snapcraft # Check that a snap was built for all architectures ls qr-server_1.0.0_{amd64,arm64,armhf,ppc64el,riscv64,s390x}.snap # Check the architecture of the qrencode library bundled in each of the snap files for arch in amd64 arm64 armhf ppc64el riscv64 s390x; do unsquashfs "qr-server_1.0.0_$arch.snap" echo "Inspecting qrencode binary from within snap built for $arch" file squashfs-root/usr/bin/qrencode rm -rf squashfs-root done ``` ### Environment I am running snapcraft 8.7.2 (13860) from latest/stable on Ubuntu 24.10, using lxd as a backend. ### snapcraft.yaml ```yaml name: qr-server base: core24 version: '1.0.0' summary: A server to generate QR codes description: | Listen on a port and serve QR codes encoding data from the path or the request body. For example, if running on localhost and port 5000, then both of the following are supported ways to generate a QR code encoding the string "hello-there": - `curl http://localhost:5000/hello-there` - `curl --data-binary 'hello-there' http://localhost:5000` If the path following the hostname is non-empty, that will be used as the data to encode, and the request body will be ignored. By default, listens on port 5000. To listen on another port, use `sudo snap set qr-server port=<port>`, with `<port>` being any available port number. Both GET and POST requests are accepted and may be used interchangeably. title: QR Server icon: icon.png license: MIT source-code: https://github.com/olivercalder/qr-server website: https://github.com/olivercalder/qr-server issues: https://github.com/olivercalder/qr-server/issues platforms: amd64: build-on: [amd64, arm64] arm64: build-on: [amd64, arm64] armhf: build-on: [amd64, arm64, armhf] ppc64el: build-on: [amd64, arm64, ppc64el] riscv64: build-on: [amd64, arm64, riscv64] s390x: build-on: [amd64, arm64, s390x] grade: stable confinement: strict parts: qr-server: plugin: dump source: bin/ organize: qr-server: bin/qr-server qr_server.py: bin/qr_server.py stage-packages: - qrencode python-dependencies: plugin: python source: . python-packages: - flask apps: qr-server: command: bin/qr-server daemon: simple plugs: - network-bind ``` ### Relevant log output ```shell Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_amd64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_arm64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_armhf.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_ppc64el.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_riscv64.snap Pulling qr-server :: Downloading package: libqrencode4/snap/snapcraft/13860/lib/python3.12/site-packages/apt/package.py:810: Warning: W:Download is performed unsandboxed as root as file '/root/.cache/snapcraft/download/libqrencode4_4.1.1-1build2_amd64.deb' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied) acq.run() Packed qr-server_1.0.0_s390x.snap ``` ### Additional context _No response_

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed not a bug Closed as expected behavior, not a bug: mr-cal explained build-on uses build-on arch for build/stage-packages; cross-compiling requires :$CRAFT_ARCH_BUILD_FOR suffixes plus ubuntu-ports repo. Later documented in craft-a-cross-compiled-app guide.
stealth/space-bunny-alpha — closed not a bug Closed as expected behavior, not a bug: cross-compilation must explicitly use :$CRAFT_ARCH_BUILD_FOR packages. A cross-compilation guide is now published.
qwen/qwen3.6-35b-a3b — — Closed as expected behavior. Snapcraft uses the build-on architecture for stage-packages. Cross-compilation requires suffixing packages with :$CRAFT_ARCH_BUILD_FOR. Correct usage is now documented in the official Snapcraft guides.
qwen/qwen3.6-35b-a3b — — Closed as expected behavior. Maintainer clarified local cross-compilation requires suffixing stage packages with :$CRAFT_ARCH_BUILD_FOR and adding the ubuntu ports repository. Documentation gap addressed with official guides.
qwen3.6-35b-a3b-mtp-q6 — — Closed as expected behavior. Maintainers clarified that local cross-compilation requires suffixing stage packages with :$CRAFT_ARCH_BUILD_FOR and adding the ubuntu ports repository. The correct workflow is now documented in the official Snapcraft docs.

Update history

No update history recorded yet.

Related work

  • Related To: canonical/snapcraft#5440 (confidence 55%)

    Docs reorganisation commit 47574a240 that added the cross-compiled app how-to page cited in the closing comment.

  • Related To: canonical/snapcraft#5265 (confidence 40%)

    fix(remotebuild): don't fail when cross-compiling (commit f94da8f13), adjacent cross-compilation handling.

Related issues

Issue Project State Summary Similarity
#1921418 wrong architecture of stage-snaps when cross building snapcraft (launchpad) open Cross-building downloads host-architecture stage-snaps instead of build-for artifacts; the acknowledged craft-415 bug appears unresolved in current craft-parts.
77%
#4542 When cross-compiling incorrect architecture `stage-packages` are fetched snapcraft closed Closed as expected behavior: stage packages use the build architecture; cross-builds require target-qualified package names and repositories.
73%
#4780 remote build: fails with some architecture definitions snapcraft closed Remote-build architecture handling was fixed by merged PR canonical/snapcraft#4793 (commit 53ec55cef), with companion support in canonical/craft-application#343.
71%
#1885150 Cannot use --build-on, architecture list is already set in snapcraft.yaml. snapcraft (launchpad) closed Resolved in Snapcraft 8.7.0: remote-build added --build-for filtering for projects defining architectures or platforms, replacing the unusable --build-on workaround.
70%