← Back to issue list

remote-build does not exit with error upon partially successful builds

View original Github issue

Metadata

Project
craft-application
Number
#989
Type
issue
State
open
Author
zhijie-yang
Labels
Created
Updated
Closed

Current evaluation

remote-build exits 0 even when some architectures fail on Launchpad, because _monitor_and_complete unconditionally returns 0. Unlabeled, no maintainer response, 227 days old; bug still present in current code.

Suggested action: needs triage

Reason: The issue is a genuine bug with a clear root cause (unconditional `return 0` at remote.py:237) and reproduction steps, but it has no labels, no assignee, and zero maintainer comments in 227 days, so it has never been triaged. The bug is still present in current code (verified: `_monitor_and_complete` returns 0 unconditionally), so it should not be closed as stale or not-a-bug. A related snapcraft issue (#4783) was fixed for a different failure mode (missing artifacts), but this partial-failure exit-code case remains unfixed.

Impact: 55 Quick Win: 38.5 Staleness: 70 Complexity: 30 Confidence: 75 Support Request: 5

Issue body

### Bug Description When building a rock that supports multiple architectures with `rockcraft remote-build`, the run exits with 0 return code even if the build is only partially successful (i.e., the builds of some architectures failed early without exceeding the time limit). The cause is the `return 0` [in the code](https://github.com/canonical/craft-application/blob/5e0eda6ad7606c35008f104bc13e039bdd16bb74/craft_application/commands/remote.py#L237). We should return a non-zero exit code if not all the architectures are successfully built. ### To Reproduce Use any `rockcraft.yaml` file that has more than one architecture defined in `platforms`, e.g. ``` platforms: amd64: arm64: armhf: ppc64el: riscv64: s390x: ``` When the build runs in launchpad, cancel one or more builds in the LP project. ### part yaml ```shell ``` ### Relevant log output ```shell 21:04:27 + rockcraft remote-build -v --launchpad-accept-public-upload 21:04:30 Starting rockcraft, version 1.16.0 21:04:30 Logging execution to '/home/jenkins/.local/state/rockcraft/log/rockcraft-20260112-210430.011354.log' 21:04:30 remote-build is experimental and is subject to change. Use with caution. 21:04:31 Starting new build. It may take a while to upload large projects. 21:04:37 Enumerating objects: 34, done. 21:04:37 Counting objects: 2% (1/34) 21:04:37 Counting objects: 5% (2/34) 21:04:37 Counting objects: 8% (3/34) 21:04:37 Counting objects: 11% (4/34) 21:04:37 Counting objects: 14% (5/34) 21:04:37 Counting objects: 17% (6/34) 21:04:37 Counting objects: 20% (7/34) 21:04:37 Counting objects: 23% (8/34) 21:04:37 Counting objects: 26% (9/34) 21:04:37 Counting objects: 29% (10/34) 21:04:37 Counting objects: 32% (11/34) 21:04:37 Counting objects: 35% (12/34) 21:04:37 Counting objects: 38% (13/34) 21:04:37 Counting objects: 41% (14/34) 21:04:37 Counting objects: 44% (15/34) 21:04:37 Counting objects: 47% (16/34) 21:04:37 Counting objects: 50% (17/34) 21:04:37 Counting objects: 52% (18/34) 21:04:37 Counting objects: 55% (19/34) 21:04:37 Counting objects: 58% (20/34) 21:04:37 Counting objects: 61% (21/34) 21:04:37 Counting objects: 64% (22/34) 21:04:37 Counting objects: 67% (23/34) 21:04:37 Counting objects: 70% (24/34) 21:04:37 Counting objects: 73% (25/34) 21:04:37 Counting objects: 76% (26/34) 21:04:37 Counting objects: 79% (27/34) 21:04:37 Counting objects: 82% (28/34) 21:04:37 Counting objects: 85% (29/34) 21:04:37 Counting objects: 88% (30/34) 21:04:37 Counting objects: 91% (31/34) 21:04:37 Counting objects: 94% (32/34) 21:04:37 Counting objects: 97% (33/34) 21:04:37 Counting objects: 100% (34/34) 21:04:37 Counting objects: 100% (34/34), done. 21:04:37 Delta compression using up to 4 threads 21:04:37 Compressing objects: 4% (1/24) 21:04:37 Compressing objects: 8% (2/24) 21:04:37 Compressing objects: 12% (3/24) 21:04:37 Compressing objects: 16% (4/24) 21:04:37 Compressing objects: 20% (5/24) 21:04:37 Compressing objects: 25% (6/24) 21:04:37 Compressing objects: 29% (7/24) 21:04:37 Compressing objects: 33% (8/24) 21:04:37 Compressing objects: 37% (9/24) 21:04:37 Compressing objects: 41% (10/24) 21:04:37 Compressing objects: 45% (11/24) 21:04:37 Compressing objects: 50% (12/24) 21:04:37 Compressing objects: 54% (13/24) 21:04:37 Compressing objects: 58% (14/24) 21:04:37 Compressing objects: 62% (15/24) 21:04:37 Compressing objects: 66% (16/24) 21:04:37 Compressing objects: 70% (17/24) 21:04:37 Compressing objects: 75% (18/24) 21:04:37 Compressing objects: 79% (19/24) 21:04:37 Compressing objects: 83% (20/24) 21:04:37 Compressing objects: 87% (21/24) 21:04:37 Compressing objects: 91% (22/24) 21:04:37 Compressing objects: 95% (23/24) 21:04:37 Compressing objects: 100% (24/24) 21:04:37 Compressing objects: 100% (24/24), done. 21:04:37 Writing objects: 2% (1/34) 21:04:37 Writing objects: 5% (2/34) 21:04:37 Writing objects: 8% (3/34) 21:04:37 Writing objects: 11% (4/34) 21:04:37 Writing objects: 14% (5/34) 21:04:37 Writing objects: 17% (6/34) 21:04:37 Writing objects: 20% (7/34) 21:04:37 Writing objects: 23% (8/34) 21:04:37 Writing objects: 26% (9/34) 21:04:37 Writing objects: 29% (10/34) 21:04:37 Writing objects: 32% (11/34) 21:04:37 Writing objects: 35% (12/34) 21:04:37 Writing objects: 38% (13/34) 21:04:37 Writing objects: 41% (14/34) 21:04:37 Writing objects: 44% (15/34) 21:04:37 Writing objects: 52% (18/34) 21:04:39 Writing objects: 52% (18/34), 2.95 MiB | 2.13 MiB/s 21:04:40 Writing objects: 52% (18/34), 13.86 MiB | 5.73 MiB/s 21:04:41 Writing objects: 52% (18/34), 27.73 MiB | 8.08 MiB/s 21:04:41 Writing objects: 55% (19/34), 27.73 MiB | 8.08 MiB/s 21:04:41 Writing objects: 58% (20/34), 27.73 MiB | 8.08 MiB/s 21:04:42 Writing objects: 58% (20/34), 39.50 MiB | 8.91 MiB/s 21:04:43 Writing objects: 58% (20/34), 51.14 MiB | 10.97 MiB/s 21:04:44 Writing objects: 58% (20/34), 63.84 MiB | 12.53 MiB/s 21:04:44 Writing objects: 61% (21/34), 63.84 MiB | 12.53 MiB/s 21:04:44 Writing objects: 64% (22/34), 63.84 MiB | 12.53 MiB/s 21:04:45 Writing objects: 64% (22/34), 76.10 MiB | 12.20 MiB/s 21:04:46 Writing objects: 64% (22/34), 86.40 MiB | 11.63 MiB/s 21:04:46 Writing objects: 67% (23/34), 92.17 MiB | 11.66 MiB/s 21:04:48 Writing objects: 67% (23/34), 108.66 MiB | 11.34 MiB/s 21:04:49 Writing objects: 67% (23/34), 119.26 MiB | 10.78 MiB/s 21:04:50 Writing objects: 70% (24/34), 125.85 MiB | 11.01 MiB/s 21:04:51 Writing objects: 70% (24/34), 145.09 MiB | 11.73 MiB/s 21:04:52 Writing objects: 70% (24/34), 158.40 MiB | 12.10 MiB/s 21:04:52 Writing objects: 73% (25/34), 163.54 MiB | 12.17 MiB/s 21:04:52 Writing objects: 76% (26/34), 163.54 MiB | 12.17 MiB/s 21:04:54 Writing objects: 76% (26/34), 179.69 MiB | 11.91 MiB/s 21:04:55 Writing objects: 76% (26/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 79% (27/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 82% (28/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 85% (29/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 88% (30/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 91% (31/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 94% (32/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 97% (33/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 100% (34/34), 191.16 MiB | 11.58 MiB/s 21:04:55 Writing objects: 100% (34/34), 195.86 MiB | 10.95 MiB/s, done. 21:04:55 Total 34 (delta 9), reused 29 (delta 7), pack-reused 0 21:05:16 remote: 21:05:16 remote: Create a merge proposal for 'main' on Launchpad by visiting: 21:05:16 remote: https://code.launchpad.net/<REDACTED>/<REDACTED>-craft-remote-build/+git/<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358/+ref/main/+register-merge 21:05:16 remote: 21:05:16 To https://git.launchpad.net/<REDACTED>/<REDACTED>-craft-remote-build/+git/rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358 21:05:16 * [new branch] <REDACTED> -> main 21:06:24 Monitoring build 21:06:26 Pending: amd64, arm64, armhf, ppc64el, riscv64, s390x 21:06:58 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:07:30 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:08:02 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:08:34 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:09:05 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:09:38 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:10:09 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:10:42 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:11:13 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:11:45 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:12:17 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:12:48 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:13:19 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:13:51 Building: amd64, arm64, armhf, ppc64el, s390x; Pending: riscv64 21:14:22 Building: arm64, armhf, ppc64el, s390x; Uploading: amd64; Pending: riscv64 21:14:54 Building: arm64, armhf, ppc64el, s390x; Uploading: amd64; Pending: riscv64 21:15:26 Building: arm64, armhf, ppc64el, s390x; Succeeded: amd64; Pending: riscv64 21:15:58 Building: arm64, armhf, ppc64el, s390x; Succeeded: amd64; Pending: riscv64 21:16:29 Building: arm64, armhf, ppc64el, s390x; Succeeded: amd64; Pending: riscv64 21:17:00 Building: arm64, armhf, ppc64el, s390x; Succeeded: amd64; Pending: riscv64 21:17:32 Building: arm64, armhf, ppc64el, s390x; Succeeded: amd64; Pending: riscv64 21:18:03 Stopped: s390x; Building: arm64, armhf, ppc64el; Succeeded: amd64; Pending: riscv64 21:18:35 Stopped: s390x; Building: arm64, armhf, ppc64el; Succeeded: amd64; Pending: riscv64 21:19:06 Stopped: s390x; Building: arm64, ppc64el; Uploading: armhf; Succeeded: amd64; Pending: riscv64 21:19:38 Stopped: s390x; Building: arm64, ppc64el; Uploading: armhf; Succeeded: amd64; Pending: riscv64 21:20:11 Stopped: s390x; Building: arm64, ppc64el; Uploading: armhf; Succeeded: amd64; Pending: riscv64 21:20:43 Stopped: ppc64el, s390x; Uploading: arm64; Succeeded: amd64, armhf; Pending: riscv64 21:21:15 Stopped: ppc64el, s390x; Succeeded: amd64, arm64, armhf; Pending: riscv64 ... 00:52:59 Stopped: ppc64el, riscv64, s390x; Succeeded: amd64, arm64, armhf 00:52:59 Fetching build artifacts... 00:53:01 Downloading 3 files (--->) 00:53:09 Downloading 3 files (<---) 00:53:09 Fetching 6 build logs... 00:53:11 Downloading 6 files (--->) 00:53:11 Downloading 6 files (<---) 00:53:11 Cleaning up 00:53:12 Warning: 'entrypoint-command' is defined. This operation will result in a rock with an atypical OCI Entrypoint. While that might be acceptable for testing and personal use, it shall require prior approval before submitting to a Canonical registry namespace. 00:53:12 Build completed. 00:53:12 Log files: rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358_amd64_2026-01-13T00:53:09.txt, rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358_arm64_2026-01-13T00:53:09.txt, rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358_armhf_2026-01-13T00:53:09.txt, rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358_ppc64el_2026-01-13T00:53:09.txt, rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358_riscv64_2026-01-13T00:53:09.txt, rockcraft-<REDACTED>-0c3e9b7341a5a0a2f7f9068c52c9c358_s390x_2026-01-13T00:53:09.txt 00:53:12 Artifacts: <REDACTED>_amd64.rock, <REDACTED>_arm64.rock, <REDACTED>_armhf.rock 00:53:13 + popd # CI workflow continues below ```

Evaluation history

Date Model Scores Action Summary
qwen/qwen3.8-27b
Impact: 55
Quick Win: 38.5
Staleness: 70
Complexity: 30
Confidence: 75
Support Request: 5
needs triage remote-build exits 0 even when some architectures fail on Launchpad, because _monitor_and_complete unconditionally returns 0. Unlabeled, no maintainer response, 227 days old; bug still present in current code.
qwen/qwen3.6-35b-a3b
Staleness: 45
Complexity: 20
Confidence: 90
Support Request: 0
needs triage Bug in rockcraft remote-build where it exits with code 0 despite partial architecture failures. Unlabelled and awaiting maintainer review after 6 months of inactivity.
qwen3.6-35b-a3b-mtp-q6
Staleness: 85
Complexity: 20
Confidence: 90
Support Request: 5
needs triage remote-build exits with code 0 despite partial failures in multi-architecture builds. Points to hardcoded return 0 in remote.py. Unlabeled and inactive for 173 days with no maintainer response.
qwen3.6-35b-a3b-mtp-q6
Staleness: 45
Complexity: 20
Confidence: 90
Support Request: 0
needs triage rockcraft remote-build returns exit code 0 on partial multi-arch build failures instead of signaling an error. Points to a hardcoded return 0 in the remote command. Open 151 days, no comments, awaiting triage and a fix.

Update history

No update history recorded yet.

Related work

  • Related To: canonical/snapcraft#4783 (confidence 60%)

    Similar silent-success-on-remote-build problem, fixed in snapcraft via #4826 for missing artifacts; this craft-application issue covers the broader partial-failure exit code case.

Related issues

Issue Project State Summary Similarity
#4495 remote build: odd errors for certain `architectures:` setups snapcraft closed Remote build architecture validation errors and misleading logs were fixed via remote-builder improvements and PR 4994. Conflicting build-for flags now yield a user-friendly error, and Launchpad reporting was corrected. Closed.
71%