remote build: hangs when logging into Launchpad
Metadata
Current evaluation
Resolved via craft-application#337 with improved Launchpad API retry logic. The CI-specific login hang was explicitly rejected since remote-build is unsupported in CI environments.
Suggested action: —
No scores available.
Issue body
### Bug Description
When running `snapcraft remote-build` the process sometimes hangs forever on:
```
remote-build is experimental and is subject to change. Use with caution.
```
[(Example)](https://github.com/canonical/checkbox/actions/runs/9221319408/job/25370115911)
Re-running the build manually offline (enabling trace level logging) what seems to happen is that the last log line before the process gets stuck is
```
2024-05-24 13:55:52.731 Emitter: Pausing control of the terminal
```
Re-trying multiple times, after this line what happens is that this line directly precedes the opening of the oauth page on launchpad. There may be some issue/race condition here?
EDIT: this seems to happen even after storing the credentials in `~/.local/share/snapcraft/launchpad-credentials `
EDIT2: this seems to happen sometimes in other spots, the one above is the most likely but it seems to be connected to this function somehow: (I stumbled upon this issue in a completely different step of the remote build, same function was blocking after Ctrl+C, all of the log below!)
```
2024-05-24 17:13:52.498 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
2024-05-24 17:13:52.498 self._sslobj.do_handshake()
```
### To Reproduce
I'm currently running
```
snapcraft remote-build --build-for armhf --launchpad-accept-public-upload --build-id checkbox24_build0 --verbosity trace
```
Without any credentials file in the launchpad-credentials, seems to fail about 50% of the times
### Environment
ArchLinux, snapcraft 8.2.8
CI/CD on ubuntu/latest (alegedly, I couldn't interrupt it to get the stacktrace but it is probably the same issue)
### snapcraft.yaml
```shell
https://github.com/canonical/checkbox/blob/main/checkbox-core-snap/series24/snap/snapcraft.yaml
(seems to be unrelated to the snapcraft.yaml)
```
### Relevant log output
```shell
2024-05-24 15:07:23.834 Starting snapcraft, version 8.2.8
2024-05-24 15:07:23.834 Log verbosity level set to BRIEF
2024-05-24 15:07:23.871 Raw pre-parsed sysargs: args={'help': False, 'verbose': False, 'quiet': False, 'verbosity': 'trace', 'version': False} filtered=['remote-build', '--build-for', 'armhf', '--launchpad-accept-public-upload', '--project', 'checkbox24_build0']
2024-05-24 15:07:23.871 General parsed sysargs: command='remote-build' args=['--build-for', 'armhf', '--launchpad-accept-public-upload', '--project', 'checkbox24_build0']
2024-05-24 15:07:23.871 Configuring application...
2024-05-24 15:07:23.872 Preparing application...
2024-05-24 15:07:23.873 Command parsed sysargs: Namespace(recover=False, launchpad_accept_public_upload=True, launchpad_timeout=0, status=False, build_id=None, platform=None, build_for='armhf', project='checkbox24_build0')
2024-05-24 15:07:23.873 Build plan: platform=None, build_for=armhf
2024-05-24 15:07:23.873 Loading project file '/home/h25/prj/canonical/checkbox_clean/snap/snapcraft.yaml'
2024-05-24 15:07:23.967 CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
2024-05-24 15:07:23.968 CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
2024-05-24 15:07:23.968 CRAFT_ARCH_TRIPLET is deprecated, use CRAFT_ARCH_TRIPLET_BUILD_{ON|FOR}
2024-05-24 15:07:23.972 Processing grammar (on amd64 for armhf)
2024-05-24 15:07:23.972 Processing grammar for plugin: dump
2024-05-24 15:07:23.972 Processing grammar for source: .
2024-05-24 15:07:23.973 Processing grammar for override-pull: craftctl default
# version.txt created by prepare.sh
export version=`cat $CRAFT_PART_SRC/version.txt`
[ $version ] || exit 1
craftctl set version=$version
2024-05-24 15:07:23.973 Processing grammar for stage: ['version.txt']
2024-05-24 15:07:23.973 Processing grammar for plugin: nil
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['fwts']
2024-05-24 15:07:23.973 Processing grammar for after: ['version-calculator']
2024-05-24 15:07:23.973 Processing grammar for plugin: nil
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['stress-ng']
2024-05-24 15:07:23.973 Processing grammar for after: ['fwts']
2024-05-24 15:07:23.973 Processing grammar for plugin: nil
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['acpica-tools']
2024-05-24 15:07:23.973 Processing grammar for after: ['stress-ng']
2024-05-24 15:07:23.973 Processing grammar for plugin: python
2024-05-24 15:07:23.973 Processing grammar for source: checkbox-support
2024-05-24 15:07:23.973 Processing grammar for source-type: local
2024-05-24 15:07:23.973 Processing grammar for stage-packages: ['python3-bluez', 'python3-pyparsing', 'python3-requests-unixsocket', 'python3-systemd', 'libpython3-stdlib', 'libpython3.12-stdlib', 'libpython3.12-minimal', 'python3-pip', 'python3-setuptools', 'python3-wheel', 'python3-venv', 'python3-minimal', 'python3-pkg-resources', 'python3.12-minimal']
2024-05-24 15:07:23.973 Processing grammar for build-packages: ['libbluetooth-dev', 'python3-dev']
2024-05-24 15:07:23.973 Not processing grammar for non-grammar enabled keyword python-packages
2024-05-24 15:07:23.973 Processing grammar for after: ['acpi-tools']
2024-05-24 15:07:23.974 Processing grammar for stage: ['-pyvenv.cfg', '-bin/activate*', '-**/RECORD', '-**/__pycache__', '-debian']
2024-05-24 15:07:23.974 Processing grammar for build-environment: [{'C_INCLUDE_PATH': '/usr/include/python3.12'}, {'PYTHONPATH': '$CRAFT_PART_INSTALL/usr/lib/python3/dist-packages:${PYTHONPATH:-}'}, {'SETUPTOOLS_SCM_PRETEND_VERSION': '$(cat /home/h25/prj/canonical/checkbox_clean/stage/version.txt)'}]
2024-05-24 15:07:23.974 Processing grammar for override-build: craftctl default
2024-05-24 15:07:23.974 Processing grammar for plugin: python
2024-05-24 15:07:23.974 Processing grammar for source: checkbox-ng
2024-05-24 15:07:23.974 Processing grammar for source-type: local
2024-05-24 15:07:23.974 Processing grammar for build-packages: ['zlib1g-dev', 'build-essential']
2024-05-24 15:07:23.974 Processing grammar for stage-packages: ['perl', 'python3-markupsafe', 'python3-jinja2', 'python3-packaging', 'python3-padme', 'python3-requests-oauthlib', 'python3-urwid', 'python3-xlsxwriter', 'libpython3-stdlib', 'libpython3.12-stdlib', 'libpython3.12-minimal', 'python3-pip', 'python3-setuptools', 'python3-wheel', 'python3-venv', 'python3-minimal', 'python3-pkg-resources', 'python3.12-minimal']
2024-05-24 15:07:23.974 Not processing grammar for non-grammar enabled keyword python-packages
2024-05-24 15:07:23.974 Processing grammar for after: ['checkbox-support']
2024-05-24 15:07:23.974 Processing grammar for stage: ['-pyvenv.cfg', '-bin/activate*', '-**/RECORD', '-**/__pycache__', '-debian']
2024-05-24 15:07:23.974 Processing grammar for build-environment: [{'PYTHONPATH': '$CRAFT_PART_INSTALL/usr/lib/python3/dist-packages:${PYTHONPATH:-}'}, {'READTHEDOCS': 'True'}, {'SETUPTOOLS_SCM_PRETEND_VERSION': '$(cat /home/h25/prj/canonical/checkbox_clean/stage/version.txt)'}]
2024-05-24 15:07:23.974 Processing grammar for override-build: craftctl default
2024-05-24 15:07:23.974 Processing grammar for plugin: dump
2024-05-24 15:07:23.974 Processing grammar for source: providers/resource
2024-05-24 15:07:23.974 Processing grammar for source-type: local
2024-05-24 15:07:23.974 Processing grammar for stage-packages: ['cpu-checker', 'dpkg', 'dmidecode', 'libjson-xs-perl', 'pciutils', 'python3-requests-unixsocket', 'smartmontools']
2024-05-24 15:07:23.974 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.974 Processing grammar for override-build: cd src && autoreconf -i && cd -
export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-resource --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.975 Processing grammar for build-packages: ['autoconf', 'automake', 'libnl-3-dev', 'libnl-genl-3-dev', 'pkg-config']
2024-05-24 15:07:23.975 Processing grammar for after: ['checkbox-ng']
2024-05-24 15:07:23.975 Processing grammar for plugin: dump
2024-05-24 15:07:23.975 Processing grammar for source: providers/base
2024-05-24 15:07:23.975 Processing grammar for source-type: local
2024-05-24 15:07:23.975 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.975 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-base --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.975 Processing grammar for override-prime: # needed because else this will trigger the store autoreviewer and be autorejected
# cleaning this bit should not cause any issue but if it does do not only remove this line
# contact also the store/update the package because else this will always trigger
# a manual review
for libamd_path in $(find .. -name "libamdhip64.so.5*"); do
echo "Clearing execstack on $libamd_path";
execstack --clear-execstack "$libamd_path" || exit 1
done;
craftctl default
2024-05-24 15:07:23.975 Processing grammar for stage-packages: ['python3-opencv', 'bc', 'bonnie++', 'cryptsetup-bin', 'dbus', 'debsums', 'dmidecode', 'dmsetup', 'efibootmgr', 'ethtool', 'freeipmi-tools', 'fswebcam', 'gir1.2-cheese-3.0', 'gir1.2-clutter-1.0', 'gir1.2-gst-plugins-base-1.0', 'gir1.2-gudev-1.0', 'glmark2-es2', 'gnome-screenshot', 'gstreamer1.0-tools', 'gstreamer1.0-pulseaudio', {'on amd64': ['hdapsd']}, 'hdparm', 'i2c-tools', 'ipmitool', 'iperf', 'iperf3', 'iw', 'jq', 'kmod', 'libasound2', 'libcap2-bin', 'libfdt1', 'libsvm3', 'lsb-release', 'lshw', 'mesa-utils', 'mokutil', 'net-tools', 'nmap', 'nux-tools', 'nvme-cli', 'obexftp', 'parted', 'pciutils', 'pulseaudio-utils', 'pyotherside-doc', 'pyotherside-tests', 'qml-module-io-thp-pyotherside', 'qml6-module-io-thp-pyotherside', 'python3-dbus', 'python3-evdev', 'python3-gi', 'python3-natsort', 'python3-pil', 'python3-psutil', 'python3-pyqrcode', 'python3-serial', 'python3-yaml', 'python3-zbar', 'qml-module-qtquick-controls', 'qml-module-qtquick-layouts', 'qmlscene', 'smartmontools', 'usbutils', 'util-linux', 'uuid-runtime', 'wget', 'wmctrl', 'xz-utils', 'rt-tests', 'mdadm', {'on armhf': ['python3-rpi.gpio']}, {'on arm64': ['python3-rpi.gpio']}]
2024-05-24 15:07:23.975 Processing grammar for build-packages: ['execstack', 'libasound2-dev', 'libcap-dev']
2024-05-24 15:07:23.975 Processing grammar for organize: {'usr/lib/lib*.so*': 'usr/lib/arm-linux-gnueabihf/'}
2024-05-24 15:07:23.975 Processing grammar for after: ['checkbox-provider-resource']
2024-05-24 15:07:23.975 Processing grammar for plugin: dump
2024-05-24 15:07:23.976 Processing grammar for source: providers/docker
2024-05-24 15:07:23.976 Processing grammar for source-type: local
2024-05-24 15:07:23.976 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.976 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-docker --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.976 Processing grammar for stage-packages: ['apache2-utils']
2024-05-24 15:07:23.976 Processing grammar for after: ['checkbox-provider-base']
2024-05-24 15:07:23.976 Processing grammar for plugin: dump
2024-05-24 15:07:23.976 Processing grammar for source: providers/tpm2
2024-05-24 15:07:23.976 Processing grammar for source-type: local
2024-05-24 15:07:23.976 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.976 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-tpm2 --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.976 Processing grammar for stage-packages: ['clevis-tpm2']
2024-05-24 15:07:23.976 Processing grammar for after: ['checkbox-provider-docker']
2024-05-24 15:07:23.976 Processing grammar for plugin: dump
2024-05-24 15:07:23.976 Processing grammar for source: providers/iiotg
2024-05-24 15:07:23.976 Processing grammar for source-type: local
2024-05-24 15:07:23.976 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-iiotg --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.977 Processing grammar for after: ['checkbox-provider-tpm2']
2024-05-24 15:07:23.977 Processing grammar for plugin: dump
2024-05-24 15:07:23.977 Processing grammar for source: providers/sru
2024-05-24 15:07:23.977 Processing grammar for source-type: local
2024-05-24 15:07:23.977 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-sru --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.977 Processing grammar for after: ['checkbox-provider-iiotg']
2024-05-24 15:07:23.977 Processing grammar for plugin: dump
2024-05-24 15:07:23.977 Processing grammar for source: providers/gpgpu
2024-05-24 15:07:23.977 Processing grammar for source-type: local
2024-05-24 15:07:23.977 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-gpgpu --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.977 Processing grammar for after: ['checkbox-provider-sru']
2024-05-24 15:07:23.977 Processing grammar for plugin: dump
2024-05-24 15:07:23.977 Processing grammar for source: providers/certification-client
2024-05-24 15:07:23.977 Processing grammar for source-type: local
2024-05-24 15:07:23.977 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.977 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-certification-client --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.978 Processing grammar for after: ['checkbox-provider-gpgpu']
2024-05-24 15:07:23.978 Processing grammar for plugin: dump
2024-05-24 15:07:23.978 Processing grammar for source: providers/certification-server
2024-05-24 15:07:23.978 Processing grammar for source-type: local
2024-05-24 15:07:23.978 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.978 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-certification-server --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.978 Processing grammar for after: ['checkbox-provider-certification-client']
2024-05-24 15:07:23.978 Processing grammar for plugin: dump
2024-05-24 15:07:23.978 Processing grammar for source: providers/tutorial
2024-05-24 15:07:23.978 Processing grammar for source-type: local
2024-05-24 15:07:23.978 Processing grammar for build-environment: [{'PYTHONPYCACHEPREFIX': '/tmp'}]
2024-05-24 15:07:23.978 Processing grammar for override-build: export PYTHONPATH=/home/h25/prj/canonical/checkbox_clean/stage/lib/python3.12/site-packages:/home/h25/prj/canonical/checkbox_clean/stage/usr/lib/python3/dist-packages
for path in $(find "/home/h25/prj/canonical/checkbox_clean/stage/providers/" -mindepth 1 -maxdepth 1 -type d); do export PROVIDERPATH=$path${PROVIDERPATH:+:$PROVIDERPATH}; done
python3 manage.py validate
python3 manage.py build
python3 manage.py install --layout=relocatable --prefix=/providers/checkbox-provider-tutorial --root="$CRAFT_PART_INSTALL"
2024-05-24 15:07:23.978 Processing grammar for after: ['checkbox-provider-base']
2024-05-24 15:07:23.978 Processing grammar for source: https://github.com/maxwellainatchi/gnome-randr-rust.git
2024-05-24 15:07:23.978 Processing grammar for plugin: rust
2024-05-24 15:07:23.978 Not processing grammar for non-grammar enabled keyword rust-channel
2024-05-24 15:07:23.978 Processing grammar for build-packages: ['libdbus-1-dev']
2024-05-24 15:07:23.979 Processing grammar for plugin: nil
2024-05-24 15:07:23.979 Processing grammar for stage-packages: [{'on armhf': ['libraspberrypi0']}, {'on arm64': ['libraspberrypi0']}]
2024-05-24 15:07:23.979 Processing grammar for organize: {'usr/lib/lib*.so': 'usr/lib/arm-linux-gnueabihf/'}
2024-05-24 15:07:23.979 Processing grammar for plugin: nil
2024-05-24 15:07:23.979 Processing grammar for source: .
2024-05-24 15:07:23.979 Processing grammar for override-pull: craftctl default
wget https://raw.githubusercontent.com/snapcore/snapd/master/include/lk/snappy_boot_common.h
wget https://raw.githubusercontent.com/snapcore/snapd/master/include/lk/snappy_boot_v2.h
wget https://raw.githubusercontent.com/kubiko/dragonboard-gadget/20-lk/snap-boot-sel/lk-boot-env.c
2024-05-24 15:07:23.979 Processing grammar for build-packages: ['wget', {'on amd64 to armhf': ['gcc-arm-linux-gnueabihf:amd64']}, {'on amd64 to arm64': ['gcc-aarch64-linux-gnu:amd64']}]
2024-05-24 15:07:23.979 Processing grammar for override-build: mkdir -p ${CRAFT_PART_INSTALL}/bin
if [ "arm-linux-gnueabihf" = "arm-linux-gnueabihf" ]; then
arm-linux-gnueabihf-gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${CRAFT_PART_INSTALL}/bin/lk-boot-env
elif [ "arm-linux-gnueabihf" = "aarch64-linux-gnu" ]; then
aarch64-linux-gnu-gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${CRAFT_PART_INSTALL}/bin/lk-boot-env
else
# native build
gcc lk-boot-env.c -I/usr/include/ -Iapp/aboot -o ${CRAFT_PART_INSTALL}/bin/lk-boot-env
fi
2024-05-24 15:07:23.979 Processing grammar for plugin: nil
2024-05-24 15:07:23.979 Processing grammar for build-environment: [{'RUNTIME_VERSION': '$(cat /home/h25/prj/canonical/checkbox_clean/stage/version.txt)'}]
2024-05-24 15:07:23.979 Processing grammar for override-build: craftctl default
echo "checkbox-runtime:" >> $CRAFT_PART_INSTALL/parts_meta_info
echo "$RUNTIME_VERSION" >> $CRAFT_PART_INSTALL/parts_meta_info
2024-05-24 15:07:23.979 Processing grammar for after: ['checkbox-provider-certification-server']
2024-05-24 15:07:23.979 Processing grammar for plugin: dump
2024-05-24 15:07:23.979 Processing grammar for source: config/
2024-05-24 15:07:23.982 Running snapcraft remote-build on host
2024-05-24 15:07:23.984 Setting up RequestService
2024-05-24 15:07:23.984 Emitter: Pausing control of the terminal
2024-05-24 15:09:19.263 Emitter: Resuming control of the terminal
2024-05-24 15:09:19.263 Interrupted.
2024-05-24 15:09:19.265 Traceback (most recent call last):
2024-05-24 15:09:19.265 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/application.py", line 517, in run
2024-05-24 15:09:19.265 return_code = dispatcher.run() or 0
2024-05-24 15:09:19.265 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run
2024-05-24 15:09:19.266 return self._loaded_command.run(self._parsed_command_args)
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/commands/base.py", line 196, in run
2024-05-24 15:09:19.266 result = self._run(parsed_args, **kwargs) or result
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 174, in _run
2024-05-24 15:09:19.266 self._services.remote_build.set_project(parsed_args.project)
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/service_factory.py", line 95, in __getattr__
2024-05-24 15:09:19.266 instance.setup()
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 76, in setup
2024-05-24 15:09:19.266 self.lp = self._get_lp_client()
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 240, in _get_lp_client
2024-05-24 15:09:19.266 return launchpad.Launchpad.login(
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/launchpad/launchpad.py", line 98, in login
2024-05-24 15:09:19.266 launchpadlib.launchpad.Launchpad.login_with(
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 700, in login_with
2024-05-24 15:09:19.266 return cls._authorize_token_and_login(
2024-05-24 15:09:19.266 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 464, in _authorize_token_and_login
2024-05-24 15:09:19.267 return cls(
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 230, in __init__
2024-05-24 15:09:19.267 super(Launchpad, self).__init__(
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 511, in __init__
2024-05-24 15:09:19.267 self._wadl = self._browser.get_wadl_application(self._root_uri)
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 502, in get_wadl_application
2024-05-24 15:09:19.267 response, content = self._request(url, media_type=wadl_type)
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 441, in _request
2024-05-24 15:09:19.267 response, content = self._request_and_retry(
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 400, in _request_and_retry
2024-05-24 15:09:19.267 response, content = self._connection.request(
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1692, in request
2024-05-24 15:09:19.267 (response, new_content) = self._request(
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 144, in _request
2024-05-24 15:09:19.267 response, content = super(LaunchpadOAuthAwareHttp, self)._request(
2024-05-24 15:09:19.267 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 204, in _request
2024-05-24 15:09:19.268 return super(RestfulHttp, self)._request(
2024-05-24 15:09:19.268 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1444, in _request
2024-05-24 15:09:19.268 (response, content) = self._conn_request(conn, request_uri, method, body, headers)
2024-05-24 15:09:19.268 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1366, in _conn_request
2024-05-24 15:09:19.268 conn.connect()
2024-05-24 15:09:19.268 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1158, in connect
2024-05-24 15:09:19.268 self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
2024-05-24 15:09:19.268 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
2024-05-24 15:09:19.268 return self.sslsocket_class._create(
2024-05-24 15:09:19.268 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1100, in _create
2024-05-24 15:09:19.268 self.do_handshake()
2024-05-24 15:09:19.268 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
2024-05-24 15:09:19.268 self._sslobj.do_handshake()
2024-05-24 15:09:19.268 KeyboardInterrupt
2024-05-24 15:09:19.268 Full execution log: '/home/h25/.local/state/snapcraft/log/snapcraft-20240524-150723.826893.log'
```
Log for EDIT2:
```
2024-05-24 17:06:58.985 remote: Resolving deltas: 100% (55063/55063), done. \ (413.4s) ^2024-05-24 17:06:58.985 remote: Resolving deltas: 100% (55063/55063), done.
2024-05-24 17:13:52.493 Interrupted.
2024-05-24 17:13:52.496 Traceback (most recent call last):
2024-05-24 17:13:52.496 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1396, in _conn_request
2024-05-24 17:13:52.496 response = conn.getresponse()
2024-05-24 17:13:52.496 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/http/client.py", line 1375, in getresponse
2024-05-24 17:13:52.496 response.begin()
2024-05-24 17:13:52.496 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/http/client.py", line 318, in begin
2024-05-24 17:13:52.496 version, status, reason = self._read_status()
2024-05-24 17:13:52.496 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/http/client.py", line 287, in _read_status
2024-05-24 17:13:52.496 raise RemoteDisconnected("Remote end closed connection without"
2024-05-24 17:13:52.496 http.client.RemoteDisconnected: Remote end closed connection without response
2024-05-24 17:13:52.496
2024-05-24 17:13:52.496 During handling of the above exception, another exception occurred:
2024-05-24 17:13:52.496 Traceback (most recent call last):
2024-05-24 17:13:52.496 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/application.py", line 517, in run
2024-05-24 17:13:52.496 return_code = dispatcher.run() or 0
2024-05-24 17:13:52.496 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run
2024-05-24 17:13:52.496 return self._loaded_command.run(self._parsed_command_args)
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/commands/base.py", line 196, in run
2024-05-24 17:13:52.497 result = self._run(parsed_args, **kwargs) or result
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/snapcraft/commands/remote.py", line 235, in _run
2024-05-24 17:13:52.497 builds = builder.start_builds(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 130, in start_builds
2024-05-24 17:13:52.497 self._recipe = self._ensure_recipe(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 303, in _ensure_recipe
2024-05-24 17:13:52.497 recipe = self._get_recipe()
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 332, in _get_recipe
2024-05-24 17:13:52.497 return self.RecipeClass.get(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/craft_application/launchpad/models/recipe.py", line 288, in get
2024-05-24 17:13:52.497 lp.lp.snaps.getByName(owner=util.get_person_link(owner), name=name),
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/resource.py", line 642, in __call__
2024-05-24 17:13:52.497 response, content = self.root._browser._request(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 441, in _request
2024-05-24 17:13:52.497 response, content = self._request_and_retry(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 400, in _request_and_retry
2024-05-24 17:13:52.497 response, content = self._connection.request(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1724, in request
2024-05-24 17:13:52.497 (response, content) = self._request(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/launchpadlib/launchpad.py", line 144, in _request
2024-05-24 17:13:52.497 response, content = super(LaunchpadOAuthAwareHttp, self)._request(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/lazr/restfulclient/_browser.py", line 204, in _request
2024-05-24 17:13:52.497 return super(RestfulHttp, self)._request(
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1444, in _request
2024-05-24 17:13:52.497 (response, content) = self._conn_request(conn, request_uri, method, body, headers)
2024-05-24 17:13:52.497 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1405, in _conn_request
2024-05-24 17:13:52.497 conn.connect()
2024-05-24 17:13:52.498 File "/var/lib/snapd/snap/snapcraft/11772/lib/python3.10/site-packages/httplib2/__init__.py", line 1158, in connect
2024-05-24 17:13:52.498 self.sock = self._context.wrap_socket(sock, server_hostname=self.host)
2024-05-24 17:13:52.498 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
2024-05-24 17:13:52.498 return self.sslsocket_class._create(
2024-05-24 17:13:52.498 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1100, in _create
2024-05-24 17:13:52.498 self.do_handshake()
2024-05-24 17:13:52.498 File "/var/lib/snapd/snap/snapcraft/11772/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
2024-05-24 17:13:52.498 self._sslobj.do_handshake()
```
### Additional context
_No response_
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Resolved via craft-application#337 with improved Launchpad API retry logic. The CI-specific login hang was explicitly rejected since remote-build is unsupported in CI environments. | |
| qwen/qwen3.6-35b-a3b | — | — | Resolved via craft-application#337 with improved Launchpad API retry logic. The CI-specific hanging behavior is explicitly unsupported and will not be addressed. | |
| qwen/qwen3.6-35b-a3b | — | — | Resolved via craft-application#337 by improving Launchpad API retry logic. The CI environment hanging issue was explicitly rejected, as remote-build is unsupported in CI. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Partially resolved via craft-application#337 retry improvements. CI login hangs are explicitly rejected. Remote-build is unsupported in CI; use snapcraft.io for orchestration. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Partially resolved via craft-application retry improvements for Launchpad API calls. The CI authentication hang is rejected; remote-build is unsupported in CI environments, and users should orchestrate builds via snapcraft.io instead. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #275 remote build: retry launchpad queries | craft-application | closed | Resolved intermittent Launchpad query failures during remote builds by implementing exponential backoff retry logic. The fix was merged via canonical/craft-application#355, automatically retrying failed requests before timing out. | |
| #5960 remote build: internal error when build is cancelled on launchpad | snapcraft | open | Remote build crashes with MissingSchema when a build is cancelled on Launchpad: fetch_logs tries to download a log URL that is the string 'None'. Triaged by maintainer, upstream fix tracked in craft-application#996. | |
| #1827679 Update remote build login when Launchpad infrastructure is ready | snapcraft (launchpad) | closed | Closed without action. The planned authentication update for remote builds was abandoned after Launchpad infrastructure changes made the task obsolete. | |
| #4892 fix(remotebuild): use legacy launchpad credentials if they exist | snapcraft | merged | Merged fix enabling legacy Launchpad credential fallback for remote builds with a deprecation warning. Added build-for all support, updated spread tests, and resolved issue 4889. Requires craft-application 3.1.0+. Approved and passed CI. | |
| #2690 remote-build: error when --user is required | snapcraft | merged | Merged. Fixes a remote-build regression where the --user flag was not enforced during LaunchpadClient initialization. Refactors config handling, moves init earlier to fail fast, and updates login/push methods to use saved credentials. | |
| #4783 remote build: fail if launchpad fails to create artifacts | snapcraft | closed | Fixed via #4826. Snapcraft remote-build now exits with an error when Launchpad fails to produce artifacts due to architecture mismatches, replacing the previous silent success with explicit failure reporting and validation. |