### Bug Description We use `snapcraft remote-build` to build snaps for different architectures as part of our continuous delivery process. Sometimes, something goes wrong and the snap for one of the architectures fails to build, yet snapcraft does not return any error, but simply a `Build failed for arch ***` message, and exits with return code `0`. We then have no idea that some of the snaps have not been properly created (and pushed to the store) unless we manually check the logs or the snapstore. It would be better if snapcraft returned `1`, so that we could get a notification to investigate the issue and maybe re-trigger a build. ### To Reproduce Use `snapcraft remote-build` to build a snap on different architectures (amd64, arm64, armhf, etc.) We build a bunch of snaps, you can find the related `snapcraft.yaml` in [our git repository](https://github.com/canonical/checkbox/tree/main/checkbox-core-snap). ### Environment - Using snapcraft from 7.x/stable channel. - Launch command: `snapcraft remote-build --build-on amd64,arm64,armhf,i386 --launchpad-accept-public-upload` - Ubuntu 22.04 (using Github Actions, see [workflow](https://github.com/canonical/checkbox/actions/runs/4977536729/workflow)) ### snapcraft.yaml ```shell name: checkbox20 summary: Checkbox runtime and public providers description: "Checkbox runtime and public providers" grade: stable confinement: strict version: '2.6' base: core20 # Don't forget to add a new slot if a new provider part is added in the parts # section below. slots: provider-resource: interface: content read: - $SNAP/providers/checkbox-provider-resource provider-checkbox: interface: content read: - $SNAP/providers/checkbox-provider-base provider-docker: interface: content read: - $SNAP/providers/checkbox-provider-docker provider-tpm2: interface: content read: - $SNAP/providers/checkbox-provider-tpm2 provider-iiotg: interface: content read: - $SNAP/providers/checkbox-provider-iiotg provider-engineering-tests: interface: content read: - $SNAP/providers/checkbox-provider-engineering-tests provider-sru: interface: content read: - $SNAP/providers/checkbox-provider-sru provider-gpgpu: interface: content read: - $SNAP/providers/checkbox-provider-gpgpu provider-certification-client: interface: content read: - $SNAP/providers/checkbox-provider-certification-client provider-certification-server: interface: content read: - $SNAP/providers/checkbox-provider-certification-server checkbox-runtime: interface: content read: - / package-repositories: - type: apt ppa: hardware-certification/public parts: ################################################################################ # Upstream: https://kernel.ubuntu.com/git/hwe/fwts.git/plain/snapcraft.yaml fwts: source-tag: "V23.01.00" source-depth: 1 plugin: autotools source: git://kernel.ubuntu.com/hwe/fwts autotools-configure-parameters: - --prefix=/ stage-packages: - libfdt1 - libbsd0 - libpci3 build-packages: - gcc - make - autoconf - automake - libtool - libjson-c-dev - flex - bison - dh-autoreconf - libglib2.0-dev - libfdt-dev - libbsd-dev ################################################################################ # Upstream: https://kernel.ubuntu.com/git/cking/stress-ng.git/plain/snap/snapcraft.yaml stress-ng: source-tag: "V0.15.07" source-depth: 1 plugin: make source: https://github.com/ColinIanKing/stress-ng.git make-parameters: - STATIC=1 build-packages: - gcc - make - zlib1g-dev - libbsd-dev - libgcrypt20-dev - libkeyutils-dev - libapparmor-dev - libaio-dev - libcap-dev - libsctp-dev - libatomic1 - libjudy-dev - libjpeg-dev - libglvnd-dev - libgbm-dev - libxxhash-dev after: [fwts] ################################################################################ # Upstream: https://kernel.ubuntu.com/git/cking/snapcraft-snaps.git/tree/acpica/snapcraft.yaml acpi-tools: source-tag: "R10_20_22" source-depth: 1 plugin: make source: https://github.com/acpica/acpica.git build-packages: - gcc - make - bison - flex prime: - usr/bin/ after: [stress-ng] ################################################################################ checkbox-support: plugin: python source: checkbox-support source-type: local stage-packages: # actual requirements - python3-bluez - python3-pyparsing - python3-requests-unixsocket - python3-systemd # added to stage python: - libpython3-stdlib - libpython3.8-stdlib - libpython3.8-minimal - python3-pip - python3-setuptools - python3-wheel - python3-venv - python3-minimal - python3-distutils - python3-pkg-resources - python3.8-minimal build-packages: - libbluetooth-dev - python3-dev python-packages: - pynmea2 after: [acpi-tools] stage: - -pyvenv.cfg - -bin/activate* - -**/RECORD - -**/__pycache__ build-environment: - C_INCLUDE_PATH: /usr/include/python3.8 - PYTHONPATH: $SNAPCRAFT_PART_INSTALL/usr/lib/python3/dist-packages:${PYTHONPATH:-} ################################################################################ checkbox-ng: plugin: python source: checkbox-ng source-type: local build-packages: - zlib1g-dev - build-essential stage-packages: - python3-markupsafe - python3-jinja2 - python3-packaging - python3-padme - python3-requests-oauthlib - python3-urwid - python3-xlsxwriter # added to stage python: - libpython3-stdlib - libpython3.8-stdlib - libpython3.8-minimal - python3-pip - python3-setuptools - python3-wheel - python3-venv - python3-minimal - python3-distutils - python3-pkg-resources - python3.8-minimal python-packages: - tqdm - picamera # p-p-c dep that wouldnt install in another part after: [checkbox-support] stage: - -pyvenv.cfg - -bin/activate* - -**/RECORD - -**/__pycache__ build-environment: - PYTHONPATH: $SNAPCRAFT_PART_INSTALL/usr/lib/python3/dist-packages:${PYTHONPATH:-} - READTHEDOCS: 'True' # simplifies picamera install ################################################################################ checkbox-provider-resource: plugin: dump source: providers/resource source-type: local stage-packages: - cpu-checker - dpkg - dmidecode - libjson-xs-perl - pciutils - python3-requests-unixsocket - smartmontools override-build: | cd src && autoreconf -i && cd - export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-resource --root="$SNAPCRAFT_PART_INSTALL" build-packages: - autoconf - automake - libnl-3-dev - libnl-genl-3-dev - pkg-config after: [checkbox-ng] ################################################################################ checkbox-provider-base: plugin: dump source: providers/base source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-base --root="$SNAPCRAFT_PART_INSTALL" stage-packages: - bc - bluez-tests - bonnie++ - cryptsetup-bin - dbus - debsums - dmidecode - dmsetup - efibootmgr - ethtool - freeipmi-tools - fswebcam - gir1.2-cheese-3.0 - gir1.2-clutter-1.0 - gir1.2-gst-plugins-base-1.0 - gir1.2-gudev-1.0 - gnome-screenshot - gstreamer1.0-tools - gstreamer1.0-pulseaudio - on amd64: [hdapsd] # optional as not available for arm - hdparm - i2c-tools - ipmitool - iperf - iperf3 - iw - jq - kmod - libasound2 - libcap2-bin - libfdt1 - lsb-release - lshw - mesa-utils - mokutil - net-tools - nmap - nux-tools - nvme-cli - obexftp - parted - pciutils - pulseaudio-utils - pyotherside - python3-dbus - python3-evdev - python3-gi - python3-natsort - python3-pil - python3-psutil - python3-pyqrcode - python3-serial - python3-yaml - python3-zbar - qml-module-qtquick-controls - qml-module-qtquick-layouts - qmlscene - smartmontools - usbutils - util-linux - uuid-runtime - wget - wmctrl - xz-utils - on armhf: - python3-rpi.gpio # only in focal - on arm64: - python3-rpi.gpio # only in focal build-packages: - libasound2-dev - libcap-dev organize: usr/lib/lib*.so*: usr/lib/$SNAPCRAFT_ARCH_TRIPLET/ after: [checkbox-provider-resource] ################################################################################ checkbox-provider-docker: plugin: dump source: providers/docker source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-docker --root="$SNAPCRAFT_PART_INSTALL" stage-packages: - apache2-utils after: [checkbox-provider-base] ################################################################################ checkbox-provider-tpm2: plugin: dump source: providers/tpm2 source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-tpm2 --root="$SNAPCRAFT_PART_INSTALL" stage-packages: - clevis-tpm2 after: [checkbox-provider-docker] ################################################################################ checkbox-provider-iiotg: plugin: dump source: providers/iiotg source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-iiotg --root="$SNAPCRAFT_PART_INSTALL" after: [checkbox-provider-tpm2] ################################################################################ checkbox-provider-sru: plugin: dump source: providers/sru source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-sru --root="$SNAPCRAFT_PART_INSTALL" after: [checkbox-provider-iiotg] ################################################################################ checkbox-provider-gpgpu: plugin: dump source: providers/gpgpu source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-gpgpu --root="$SNAPCRAFT_PART_INSTALL" after: [checkbox-provider-sru] ################################################################################ checkbox-provider-certification-client: plugin: dump source: providers/certification-client source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-certification-client --root="$SNAPCRAFT_PART_INSTALL" after: [checkbox-provider-gpgpu] ################################################################################ checkbox-provider-certification-server: plugin: dump source: providers/certification-server source-type: local override-build: | export PYTHONPATH=$SNAPCRAFT_STAGE/lib/python3.8/site-packages:$SNAPCRAFT_STAGE/usr/lib/python3/dist-packages for path in $(find "$SNAPCRAFT_STAGE/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done python3 manage.py validate python3 manage.py build python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-certification-server --root="$SNAPCRAFT_PART_INSTALL" after: [checkbox-provider-certification-client] ################################################################################ glmark2: source: https://github.com/glmark2/glmark2.git plugin: meson build-packages: - libpng-dev - libjpeg-dev - libdrm-dev - libgbm-dev - libudev-dev - libx11-dev - meson - python3-distro meson-parameters: #- -Dflavors=drm-gl,drm-glesv2,wayland-gl,wayland-glesv2,x11-gl,x11-glesv2 - -Dflavors=x11-glesv2 - --prefix=/usr after: [checkbox-provider-certification-server] stage-packages: - python3-distro override-stage: | snapcraftctl stage sed -i 's|except OSError: # Command not found|except subprocess.CalledProcessError: # Command not found|g' usr/lib/python3/dist-packages/distro.py ################################################################################ rpi-ppa: plugin: nil override-pull: | sudo add-apt-repository ppa:sylvain-pineau/ppa-rpi sudo apt-get update build-packages: - software-properties-common rpi-support-binaries: after: [rpi-ppa] plugin: nil stage-packages: - on armhf: - libraspberrypi0 - on arm64: - libraspberrypi0 organize: usr/lib/lib*.so: usr/lib/$SNAPCRAFT_ARCH_TRIPLET/ ################################################################################ lk-boot-env: plugin: nil source: . override-pull: | snapcraftctl pull wget https://raw.githubusercontent.com/snapcore/snapd/master/include/lk/snappy_boot_common.h wget https://raw.githubusercontent.com/snapcore/snapd/master/include/lk/snappy_boot_v2.h wget https://raw.githubusercontent.com/kubiko/dragonboard-gadget/20-lk/snap-boot-sel/lk-boot-env.c build-packages: - wget - on amd64 to armhf: - gcc-arm-linux-gnueabihf:amd64 - on amd64 to arm64: - gcc-aarch64-linux-gnu:amd64 override-build: | mkdir -p ${SNAPCRAFT_PART_INSTALL}/bin if [ "${SNAPCRAFT_ARCH_TRIPLET}" = "arm-linux-gnueabihf" ]; then arm-linux-gnueabihf-gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${SNAPCRAFT_PART_INSTALL}/bin/lk-boot-env elif [ "${SNAPCRAFT_ARCH_TRIPLET}" = "aarch64-linux-gnu" ]; then aarch64-linux-gnu-gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${SNAPCRAFT_PART_INSTALL}/bin/lk-boot-env else # native build gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${SNAPCRAFT_PART_INSTALL}/bin/lk-boot-env fi ################################################################################ parts-meta-info: plugin: nil override-build: | snapcraftctl build echo $PWD ls -1 -d ../../* for p in `ls -d ../../*`; do if [ -d $p/src ]; then (cd $p/src if [ -d $p/src/.git ]; then (echo `basename $p`\: && git show --pretty=format:"%h%d %aN %ci%n%s%n" -q ; echo ) >> $SNAPCRAFT_PART_INSTALL/parts_meta_info fi # for additional `source-type` support, elif them here ) fi done after: [checkbox-provider-certification-server] ################################################################################ common-config: plugin: dump source: config/ ``` ### Relevant log output ```shell <Full logs available at: https://github.com/canonical/checkbox/actions/runs/4977536729/jobs/890671400> snapcraft remote-build is experimental and is subject to change - use with caution. Use --build-for instead of --build-on Using build ID snapcraft-checkbox20-74a3181e9730e14cca82b6e7dd7e447b Packaging sources for checkbox-support... Packaging sources for checkbox-ng... Packaging sources for checkbox-provider-resource... Packaging sources for checkbox-provider-base... Packaging sources for checkbox-provider-docker... Packaging sources for checkbox-provider-tpm2... Packaging sources for checkbox-provider-iiotg... Packaging sources for checkbox-provider-sru... Packaging sources for checkbox-provider-gpgpu... Packaging sources for checkbox-provider-certification-client... Packaging sources for checkbox-provider-certification-server... Packaging sources for lk-boot-env... Packaging sources for common-config... hint: Using 'master' as the name for the initial branch. This default branch name hint: is subject to change. To configure the initial branch name to use in all hint: of your new repositories, which will suppress this warning, call: hint: hint: git config --global init.defaultBranch <name> hint: hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and hint: 'development'. The just-created branch can be renamed via this command: hint: hint: git branch -m <name> Sending build data to Launchpad... (https://ce-certification-qa:<token>@git.launchpad.net/~ce-certification-qa/+git/snapcraft-checkbox20-74a3181e9730e14cca82b6e7dd7e447b/) To https://git.launchpad.net/~ce-certification-qa/+git/snapcraft-checkbox20-74a3181e9730e14cca82b6e7dd7e447b/ * [new branch] HEAD -> master If interrupted, resume with: 'snapcraft remote-build --recover --build-id snapcraft-checkbox20-74a3181e9730e14cca82b6e7dd7e447b' Building snap package for amd64, arm64, armhf, and i386. This may take some time to finish. Build status as of 2023-05-15 07:11:56.224741: arch=arm64 state=Needs building arch=armhf state=Needs building arch=amd64 state=Needs building Build status as of 2023-05-15 07:12:26.577110: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:12:56.900552: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:13:27.236014: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:13:57.556636: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:14:27.909221: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:14:58.279353: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:15:28.645802: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:15:58.973703: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:16:29.397651: arch=arm64 state=Currently building arch=armhf state=Currently building arch=amd64 state=Currently building Build status as of 2023-05-15 07:16:59.760598: arch=arm64 state=Currently building Build status as of 2023-05-15 07:36:16.432279: arch=arm64 state=Failed to build arch=amd64 state=Successfully built arch=armhf state=Currently building Build status as of 2023-05-15 07:36:47.034605: arch=arm64 state=Failed to build arch=amd64 state=Successfully built arch=armhf state=Currently building Build status as of 2023-05-15 07:37:17.445042: arch=arm64 state=Failed to build arch=amd64 state=Successfully built arch=armhf state=Currently building Build status as of 2023-05-15 07:37:47.799670: arch=arm64 state=Failed to build arch=amd64 state=Successfully built arch=armhf state=Currently building Build status as of 2023-05-15 07:38:18.291151: arch=armhf state=Successfully built arch=arm64 state=Failed to build arch=amd64 state=Successfully built Snapped checkbox20_2.6_armhf.snap Build log available at 'checkbox20_armhf.txt' Starting Snapcraft 7.3.1 Logging execution to '/home/runner/.local/state/snapcraft/log/snapcraft-20230515-071137.935145.log' Snap file not available for arch 'arm64'. Build log available at 'checkbox20_arm64.txt' Build failed for arch 'arm64'. Snapped checkbox20_2.6_amd64.snap Build log available at 'checkbox20_amd64.txt' Build complete. ``` ### Additional context _No response_