remote build: retry launchpad queries
Metadata
Current evaluation
Resolved intermittent Launchpad query failures during remote builds by implementing exponential backoff retry logic. The fix was merged via canonical/craft-application#355, automatically retrying failed requests before timing out.
Suggested action: —
No scores available.
Issue body
### Bug Description
Sometimes launchpad queries getting a project, repo or recipe fails and just needs a backoff before trying again. We can probably retry 3 times before fully failing.
### To Reproduce
Check charmcraft remote-build spread tests
### part yaml
_No response_
### Relevant log output
```shell
Total 18 (delta 0), reused 0 (delta 0), pack-reused 0
remote:
remote: Create a merge proposal for 'main' on Launchpad by visiting:
remote: https://code.launchpad.net/~lengau/lengau-craft-remote-build/+git/charmcraft-charm-9bae065def77f3d680f53bbcb0154b9e/+ref/main/+register-merge
remote:
To https://git.launchpad.net/~lengau/lengau-craft-remote-build/+git/charmcraft-charm-9bae065def77f3d680f53bbcb0154b9e
* [new branch] master -> main
charmcraft internal error: ValueError("Could not find charm recipe 'charmcraft-charm-9bae065def77f3d680f53bbcb0154b9e' in project 'lengau-craft-remote-build' with owner 'lengau'")
Full execution log: '/root/.local/state/charmcraft/log/charmcraft-20240319-013323.238305.log'
Traceback (most recent call last):
File "/snap/charmcraft/x1/bin/charmcraft", line 8, in <module>
sys.exit(main())
File "/snap/charmcraft/x1/lib/python3.10/site-packages/charmcraft/application/main.py", line 141, in main
return app.run()
File "/snap/charmcraft/x1/lib/python3.10/site-packages/craft_application/application.py", line 476, in run
return_code = dispatcher.run() or 0
File "/snap/charmcraft/x1/lib/python3.10/site-packages/craft_cli/dispatcher.py", line 487, in run
return self._loaded_command.run(self._parsed_command_args)
File "/snap/charmcraft/x1/lib/python3.10/site-packages/craft_application/commands/base.py", line 182, in run
result = self._run(parsed_args, **kwargs) or result
File "/snap/charmcraft/x1/lib/python3.10/site-packages/charmcraft/application/commands/remote.py", line 125, in _run
builds = builder.start_builds(project_dir)
File "/snap/charmcraft/x1/lib/python3.10/site-packages/craft_application/services/remotebuild.py", line 91, in start_builds
self._recipe = self._new_recipe(
File "/snap/charmcraft/x1/lib/python3.10/site-packages/charmcraft/services/remotebuild.py", line 75, in _new_recipe
return self.lp.get_recipe(
File "/snap/charmcraft/x1/lib/python3.10/site-packages/craft_application/launchpad/launchpad.py", line 161, in get_recipe
return models.CharmRecipe.get(self, name, owner, project)
File "/snap/charmcraft/x1/lib/python3.10/site-packages/craft_application/launchpad/models/recipe.py", line 398, in get
raise ValueError(
ValueError: Could not find charm recipe 'charmcraft-charm-9bae065def77f3d680f53bbcb0154b9e' in project 'lengau-craft-remote-build' with owner 'lengau'
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Resolved intermittent Launchpad query failures during remote builds by implementing exponential backoff retry logic. The fix was merged via canonical/craft-application#355, automatically retrying failed requests before timing out. | |
| qwen/qwen3.6-35b-a3b | — | — | Implemented exponential backoff and retry logic for Launchpad queries during remote builds. Resolved via merged PR #355 in canonical/craft-application. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Resolved intermittent Launchpad query failures during remote builds by implementing retry logic with exponential backoff. The fix was merged in craft-application PR #355, adjusting retry delays to handle transient API slowness. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #4821 remote build: hangs when logging into Launchpad | snapcraft | closed | Resolved via craft-application#337 with improved Launchpad API retry logic. The CI-specific login hang was explicitly rejected since remote-build is unsupported in CI environments. | |
| #2633 remote-build: increase number of launchpad start_build request attempts | snapcraft | closed | Closed without merging. The maintainer identified that the underlying issue stemmed from an uncaught backend error rather than insufficient retry attempts, rendering the proposed changes unnecessary. | |
| #352 feat: retry more launchpad queries | craft-application | closed | Merged via PR #355. Added a reusable retry() function to additional Launchpad API calls to handle transient failures, refactoring existing logic. More targets will be added as flaky exceptions are identified. | |
| #355 feat: exponential backoff for launchpad retries | craft-application | merged | Merged exponential backoff for launchpad retries. The retry strategy now doubles sleep intervals starting at two seconds, capped at five attempts and 62 seconds total. Approved by two reviewers, passed CI, and modified seven files. |