> 🤖 This issue was created by an AI agent (GitHub Copilot CLI) on behalf of a maintainer, while investigating flaky CI failures. ## Description Source handlers in `craft_parts.sources` do not retry on transient network failures during the `pull` step. This affects at least: * `TarSource`/`ZipSource`/other HTTP-based sources: `SourceHandler.download()` in `craft_parts/sources/base.py` issues a single `requests.get()` call with no retry, backoff, or retry-aware session configured: ```python # craft_parts/sources/base.py try: request = requests.get( self.source, stream=True, allow_redirects=True, timeout=3600 ) request.raise_for_status() url_utils.download_request(request, self._file) except requests.HTTPError as err: ... raise errors.HttpRequestError(...) from err except requests.RequestException as err: raise errors.NetworkRequestError(...) from err ``` * `GitSource`: `_run()` in `craft_parts/sources/git_source.py` (via `os_utils.process_run` in `craft_parts/sources/base.py`) shells out to `git clone`/`git fetch` once, with no retry. A transient failure while cloning over HTTPS (e.g. the remote dropping the connection mid-transfer) surfaces as a raw `subprocess.CalledProcessError` that isn't even recognized as a network error. In both cases, a single transient hiccup — a `5xx` from the origin server or a CDN, a reset/dropped connection, a momentary DNS/network blip — fails the entire `pull` step (and therefore the whole build) immediately, with no attempt to retry. This affects any real user of craft-parts pulling a `source:` URL (tarball, zip, or git repo), not just CI — it's simply most visible here because we run the same source-fetching integration tests very frequently. ## Examples from CI history `TarSource.download()` (`tests/integration/plugins/test_autotools.py::test_autotools_plugin`, downloads `hello_2.10.orig.tar.gz` from `launchpadlibrarian.net`/`launchpad.net`): * https://github.com/canonical/craft-parts/actions/runs/31522566940/job/93883282233 (2026-08-11) — `craft_parts.sources.errors.HttpRequestError: Cannot process request (Proxy Error: 502)` * https://github.com/canonical/craft-parts/actions/runs/30815509120/job/91692141413 (main, 2026-08-03) — `OSError: [Errno 101] Network is unreachable` → `urllib3.exceptions.NewConnectionError` → `requests.exceptions.ConnectionError` → `craft_parts.sources.errors.NetworkRequestError` * https://github.com/canonical/craft-parts/actions/runs/29457111352/job/87492627401 (main, 2026-07-15) — `ConnectionResetError: [Errno 104] Connection reset by peer` → `requests.exceptions.ConnectionError` → `craft_parts.sources.errors.NetworkRequestError` This same test has failed this way at least a dozen times across the last two months (e.g. runs 83724396417, 83776466896, 83776517670, 83804529989, 83804534004, 83845909077, 85709682080, 86151387606, 87489004187, 87761987882, 88075999228, 88496850084, 89735086830, 89769147054, 90199824275, 91136822317, 91431838329, 91431842766, 94209797821, 94239946054), all with the same underlying `craft_parts.sources.errors.{Network,HttpRequest}Error` cause and no retry. `GitSource._run()` (`tests/integration/plugins/test_go.py::test_go_use_incomplete_parts`, `git clone` of `https://go.googlesource.com/sys`): * https://github.com/canonical/craft-parts/actions/runs/28273822918/job/83776472668 (2026-06-27) — `subprocess.CalledProcessError: ... 'git', 'clone', ... 'https://go.googlesource.com/sys' ... returned non-zero exit status 128` with `fatal: expected 'packfile'` in the git output (a transient failure mid-clone over HTTPS) We currently work around this class of flakiness at the *test* level with `@pytest.mark.flaky(reruns=..., only_rerun=[...])` on individual integration tests (see `tests/integration/packages/test_chisel.py`, `tests/integration/lifecycle/test_chisel_lifecycle.py`, `tests/integration/utils/test_process.py`, and now `tests/integration/plugins/test_autotools.py`). That masks the flakiness in our own CI, but does nothing for actual users of craft-parts who hit the same transient errors when pulling sources. ## Proposed solution Add real retry-with-backoff to the source `pull` implementations, e.g.: * For HTTP(S)-based sources (`SourceHandler.download()` in `craft_parts/sources/base.py`): use a `requests.Session` with a `urllib3.util.retry.Retry` configuration mounted via `HTTPAdapter`, retrying on connection errors and a documented set of transient status codes (`429, 500, 502, 503, 504`), with a small number of retries and exponential backoff. Keep the existing behavior for non-retriable errors (e.g. `404` → `SourceNotFound`) unchanged — only retry the transient subset. * For `GitSource` (and any other subprocess-based source, e.g. via `os_utils.process_run`): retry the clone/fetch command itself a small number of times on non-zero exit, possibly limited to cases where the output matches known-transient git/network error patterns (e.g. `early EOF`, `expected 'packfile'`, `RPC failed`, connection resets), similar to how `git` itself is sometimes wrapped with retry logic in other tooling. * Make the retry count/backoff configurable (or at least centralize it) so it can be tuned without touching every source implementation individually. This would fix the flakiness at its root rather than continuing to paper over it with `pytest.mark.flaky` on every integration test that happens to pull sources over the network.