Fetch service external spread tests are failing
Metadata
Current evaluation
No evaluation has been recorded for this issue yet.
Issue body
### Bug Description
See: https://github.com/canonical/craft-application/actions/runs/23248068356/job/67582711482?pr=1027
The spread tests that use the external fetch service are failing.
They should succeed again when Fetch Service 0.16 is released to handle these:
- https://github.com/canonical/fetch-service/pull/548
- https://github.com/canonical/fetch-service/pull/579
### To Reproduce
```
spread google:ubuntu-22.04-64:tests/spread/testcraft/fetch-service-external google:ubuntu-24.04-64:tests/spread/testcraft/fetch-service-external
```
### part yaml
```shell
n/a
```
### Relevant log output
```shell
2026-03-18 14:01:05.239 * Command exit code: 100
2026-03-18 14:01:05.239 * Command output: b'Get:1 http://security.ubuntu.com/ubuntu noble-security InRelease\nGet:2 http://archive.ubuntu.com/ubuntu noble InRelease\nGet:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease\nGet:4 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [1541 kB]\nGet:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease\nGet:6 http://security.ubuntu.com/ubuntu noble-security/main Translation-en [247 kB]\nGet:7 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [977 kB]\nGet:8 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1401 kB]\nGet:9 http://security.ubuntu.com/ubuntu noble-security/universe Translation-en [219 kB]\nGet:10 http://archive.ubuntu.com/ubuntu noble/main Translation-en [513 kB]\nGet:11 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [2683 kB]\nGet:12 http://security.ubuntu.com/ubuntu noble-security/restricted Translation-en [623 kB]\nGet:13 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Packages [28.8 kB]\nGet:14 http://security.ubuntu.com/ubuntu noble-security/multiverse Translation-en [6732 B]\nGet:15 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [15.0 MB]\nGet:16 http://archive.ubuntu.com/ubuntu noble/universe Translation-en [5982 kB]\nGet:17 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [93.9 kB]\nGet:18 http://archive.ubuntu.com/ubuntu noble/restricted Translation-en [18.7 kB]\nGet:19 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [269 kB]\nGet:20 http://archive.ubuntu.com/ubuntu noble/multiverse Translation-en [118 kB]\nGet:21 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [1837 kB]\nGet:22 http://archive.ubuntu.com/ubuntu noble-updates/main Translation-en [336 kB]\nGet:23 http://archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [1566 kB]\nGet:24 http://archive.ubuntu.com/ubuntu noble-updates/universe Translation-en [319 kB]\nGet:25 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [2819 kB]\nGet:26 http://archive.ubuntu.com/ubuntu noble-updates/restricted Translation-en [652 kB]\nGet:27 http://archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [32.1 kB]\nGet:28 http://archive.ubuntu.com/ubuntu noble-updates/multiverse Translation-en [7044 B]\nGet:29 http://archive.ubuntu.com/ubuntu noble-backports/main amd64 Packages [40.4 kB]\nGet:30 http://archive.ubuntu.com/ubuntu noble-backports/main Translation-en [9208 B]\nGet:31 http://archive.ubuntu.com/ubuntu noble-backports/universe amd64 Packages [30.7 kB]\nGet:32 http://archive.ubuntu.com/ubuntu noble-backports/universe Translation-en [18.2 kB]\nGet:33 http://archive.ubuntu.com/ubuntu noble-backports/multiverse amd64 Packages [780 B]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nFetched 38.1 MB in 26s (1454 kB/s)\nReading package lists...\n'
2026-03-18 14:01:05.239 * Command standard error output: b'E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/noble-backports/multiverse/i18n/Translation-en 403 Forbidden [IP: 10.148.181.1 9988]\nE: Some index files failed to download. They have been ignored, or old ones used instead.\n'
2026-03-18 14:01:05.259 Traceback (most recent call last):
2026-03-18 14:01:05.259 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/bases/ubuntu.py", line 245, in _pre_setup_packages
2026-03-18 14:01:05.259 self._execute_run(
2026-03-18 14:01:05.259 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/base.py", line 1215, in _execute_run
2026-03-18 14:01:05.259 proc = executor.execute_run(
2026-03-18 14:01:05.259 ^^^^^^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.259 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/lxd/lxd_instance.py", line 279, in execute_run
2026-03-18 14:01:05.259 return self.lxc.exec(
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/lxd/lxc.py", line 536, in exec
2026-03-18 14:01:05.260 return runner(final_cmd, timeout=timeout, check=check, **kwargs)
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/python-runtime-core24-312/current/usr/lib/python3.12/subprocess.py", line 571, in run
2026-03-18 14:01:05.260 raise CalledProcessError(retcode, process.args,
2026-03-18 14:01:05.260 subprocess.CalledProcessError: Command '['lxc', '--project', 'testcraft', 'exec', 'local:testcraft-fetch-service-external-platform-independent-2064428', '--', 'env', 'CRAFT_MANAGED_MODE=1', 'TESTCRAFT_DEBUG=True', 'TESTCRAFT_LXD_REMOTE=local', 'TESTCRAFT_LAUNCHPAD_INSTANCE=production', 'http_proxy=***10.148.181.1:9988', 'HTTP_PROXY=***10.148.181.1:9988', 'https_proxy=***10.148.181.1:9988', 'HTTPS_PROXY=***10.148.181.1:9988', 'DEBIAN_FRONTEND=noninteractive', 'DEBCONF_NONINTERACTIVE_SEEN=true', 'DEBIAN_PRIORITY=critical', 'apt-get', 'update']' returned non-zero exit status 100.
2026-03-18 14:01:05.260
2026-03-18 14:01:05.260 The above exception was the direct cause of the following exception:
2026-03-18 14:01:05.260 Traceback (most recent call last):
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/lxd/lxd_provider.py", line 205, in launched_environment
2026-03-18 14:01:05.260 instance = launch(
2026-03-18 14:01:05.260 ^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/lxd/launcher.py", line 798, in launch
2026-03-18 14:01:05.260 _create_instance(
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/lxd/launcher.py", line 226, in _create_instance
2026-03-18 14:01:05.260 base_configuration.setup(executor=instance)
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/base.py", line 1072, in setup
2026-03-18 14:01:05.260 self._pre_setup_packages(executor=executor)
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/bases/ubuntu.py", line 252, in _pre_setup_packages
2026-03-18 14:01:05.260 raise BaseConfigurationError(
2026-03-18 14:01:05.260 craft_providers.errors.BaseConfigurationError: Failed to update apt cache.
2026-03-18 14:01:05.260 * Command that failed: 'lxc --project testcraft exec local:testcraft-fetch-service-external-platform-independent-2064428 -- env CRAFT_MANAGED_MODE=1 TESTCRAFT_DEBUG=True TESTCRAFT_LXD_REMOTE=local TESTCRAFT_LAUNCHPAD_INSTANCE=production http_proxy=***10.148.181.1:9988 HTTP_PROXY=***10.148.181.1:9988 https_proxy=***10.148.181.1:9988 HTTPS_PROXY=***10.148.181.1:9988 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical apt-get update'
2026-03-18 14:01:05.260 * Command exit code: 100
2026-03-18 14:01:05.260 * Command output: b'Get:1 http://security.ubuntu.com/ubuntu noble-security InRelease\nGet:2 http://archive.ubuntu.com/ubuntu noble InRelease\nGet:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease\nGet:4 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [1541 kB]\nGet:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease\nGet:6 http://security.ubuntu.com/ubuntu noble-security/main Translation-en [247 kB]\nGet:7 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [977 kB]\nGet:8 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1401 kB]\nGet:9 http://security.ubuntu.com/ubuntu noble-security/universe Translation-en [219 kB]\nGet:10 http://archive.ubuntu.com/ubuntu noble/main Translation-en [513 kB]\nGet:11 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [2683 kB]\nGet:12 http://security.ubuntu.com/ubuntu noble-security/restricted Translation-en [623 kB]\nGet:13 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Packages [28.8 kB]\nGet:14 http://security.ubuntu.com/ubuntu noble-security/multiverse Translation-en [6732 B]\nGet:15 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [15.0 MB]\nGet:16 http://archive.ubuntu.com/ubuntu noble/universe Translation-en [5982 kB]\nGet:17 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [93.9 kB]\nGet:18 http://archive.ubuntu.com/ubuntu noble/restricted Translation-en [18.7 kB]\nGet:19 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [269 kB]\nGet:20 http://archive.ubuntu.com/ubuntu noble/multiverse Translation-en [118 kB]\nGet:21 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [1837 kB]\nGet:22 http://archive.ubuntu.com/ubuntu noble-updates/main Translation-en [336 kB]\nGet:23 http://archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [1566 kB]\nGet:24 http://archive.ubuntu.com/ubuntu noble-updates/universe Translation-en [319 kB]\nGet:25 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [2819 kB]\nGet:26 http://archive.ubuntu.com/ubuntu noble-updates/restricted Translation-en [652 kB]\nGet:27 http://archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [32.1 kB]\nGet:28 http://archive.ubuntu.com/ubuntu noble-updates/multiverse Translation-en [7044 B]\nGet:29 http://archive.ubuntu.com/ubuntu noble-backports/main amd64 Packages [40.4 kB]\nGet:30 http://archive.ubuntu.com/ubuntu noble-backports/main Translation-en [9208 B]\nGet:31 http://archive.ubuntu.com/ubuntu noble-backports/universe amd64 Packages [30.7 kB]\nGet:32 http://archive.ubuntu.com/ubuntu noble-backports/universe Translation-en [18.2 kB]\nGet:33 http://archive.ubuntu.com/ubuntu noble-backports/multiverse amd64 Packages [780 B]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nFetched 38.1 MB in 26s (1454 kB/s)\nReading package lists...\n'
2026-03-18 14:01:05.260 * Command standard error output: b'E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/noble-backports/multiverse/i18n/Translation-en 403 Forbidden [IP: 10.148.181.1 9988]\nE: Some index files failed to download. They have been ignored, or old ones used instead.\n'
2026-03-18 14:01:05.260
2026-03-18 14:01:05.260 The above exception was the direct cause of the following exception:
2026-03-18 14:01:05.260 Traceback (most recent call last):
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/application.py", line 678, in run
2026-03-18 14:01:05.260 return_code = self._run_inner()
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/application.py", line 655, in _run_inner
2026-03-18 14:01:05.260 return_code = dispatcher.run() or os.EX_OK
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_cli/dispatcher.py", line 564, in run
2026-03-18 14:01:05.260 return self._loaded_command.run(self._parsed_command_args)
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/commands/base.py", line 222, in run
2026-03-18 14:01:05.260 result = self._run(parsed_args, **kwargs) or result
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/commands/lifecycle.py", line 630, in _run
2026-03-18 14:01:05.260 return super()._run(parsed_args=parsed_args, step_name=step_name)
2026-03-18 14:01:05.260 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2026-03-18 14:01:05.260 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/commands/lifecycle.py", line 243, in _run
2026-03-18 14:01:05.261 self._run_manager_for_build_plan(fetch_service_policy)
2026-03-18 14:01:05.261 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/commands/lifecycle.py", line 108, in _run_manager_for_build_plan
2026-03-18 14:01:05.261 provider.run_managed(build, bool(fetch_service_policy))
2026-03-18 14:01:05.261 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/services/provider.py", line 494, in run_managed
2026-03-18 14:01:05.261 with self.instance(
2026-03-18 14:01:05.261 File "/snap/python-runtime-core24-312/current/usr/lib/python3.12/contextlib.py", line 137, in __enter__
2026-03-18 14:01:05.261 return next(self.gen)
2026-03-18 14:01:05.261 ^^^^^^^^^^^^^^
2026-03-18 14:01:05.261 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_application/services/provider.py", line 221, in instance
2026-03-18 14:01:05.261 with provider.launched_environment(
2026-03-18 14:01:05.261 File "/snap/python-runtime-core24-312/current/usr/lib/python3.12/contextlib.py", line 137, in __enter__
2026-03-18 14:01:05.261 return next(self.gen)
2026-03-18 14:01:05.262 ^^^^^^^^^^^^^^
2026-03-18 14:01:05.262 File "/snap/testcraft/x1/lib/python3.12/site-packages/craft_providers/lxd/lxd_provider.py", line 221, in launched_environment
2026-03-18 14:01:05.262 raise LXDError(str(error)) from error
2026-03-18 14:01:05.262 craft_providers.lxd.errors.LXDError: Failed to update apt cache.
2026-03-18 14:01:05.262 * Command that failed: 'lxc --project testcraft exec local:testcraft-fetch-service-external-platform-independent-2064428 -- env CRAFT_MANAGED_MODE=1 TESTCRAFT_DEBUG=True TESTCRAFT_LXD_REMOTE=local TESTCRAFT_LAUNCHPAD_INSTANCE=production http_proxy=***10.148.181.1:9988 HTTP_PROXY=***10.148.181.1:9988 https_proxy=***10.148.181.1:9988 HTTPS_PROXY=***10.148.181.1:9988 DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true DEBIAN_PRIORITY=critical apt-get update'
2026-03-18 14:01:05.262 * Command exit code: 100
2026-03-18 14:01:05.262 * Command output: b'Get:1 http://security.ubuntu.com/ubuntu noble-security InRelease\nGet:2 http://archive.ubuntu.com/ubuntu noble InRelease\nGet:3 http://archive.ubuntu.com/ubuntu noble-updates InRelease\nGet:4 http://security.ubuntu.com/ubuntu noble-security/main amd64 Packages [1541 kB]\nGet:5 http://archive.ubuntu.com/ubuntu noble-backports InRelease\nGet:6 http://security.ubuntu.com/ubuntu noble-security/main Translation-en [247 kB]\nGet:7 http://security.ubuntu.com/ubuntu noble-security/universe amd64 Packages [977 kB]\nGet:8 http://archive.ubuntu.com/ubuntu noble/main amd64 Packages [1401 kB]\nGet:9 http://security.ubuntu.com/ubuntu noble-security/universe Translation-en [219 kB]\nGet:10 http://archive.ubuntu.com/ubuntu noble/main Translation-en [513 kB]\nGet:11 http://security.ubuntu.com/ubuntu noble-security/restricted amd64 Packages [2683 kB]\nGet:12 http://security.ubuntu.com/ubuntu noble-security/restricted Translation-en [623 kB]\nGet:13 http://security.ubuntu.com/ubuntu noble-security/multiverse amd64 Packages [28.8 kB]\nGet:14 http://security.ubuntu.com/ubuntu noble-security/multiverse Translation-en [6732 B]\nGet:15 http://archive.ubuntu.com/ubuntu noble/universe amd64 Packages [15.0 MB]\nGet:16 http://archive.ubuntu.com/ubuntu noble/universe Translation-en [5982 kB]\nGet:17 http://archive.ubuntu.com/ubuntu noble/restricted amd64 Packages [93.9 kB]\nGet:18 http://archive.ubuntu.com/ubuntu noble/restricted Translation-en [18.7 kB]\nGet:19 http://archive.ubuntu.com/ubuntu noble/multiverse amd64 Packages [269 kB]\nGet:20 http://archive.ubuntu.com/ubuntu noble/multiverse Translation-en [118 kB]\nGet:21 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 Packages [1837 kB]\nGet:22 http://archive.ubuntu.com/ubuntu noble-updates/main Translation-en [336 kB]\nGet:23 http://archive.ubuntu.com/ubuntu noble-updates/universe amd64 Packages [1566 kB]\nGet:24 http://archive.ubuntu.com/ubuntu noble-updates/universe Translation-en [319 kB]\nGet:25 http://archive.ubuntu.com/ubuntu noble-updates/restricted amd64 Packages [2819 kB]\nGet:26 http://archive.ubuntu.com/ubuntu noble-updates/restricted Translation-en [652 kB]\nGet:27 http://archive.ubuntu.com/ubuntu noble-updates/multiverse amd64 Packages [32.1 kB]\nGet:28 http://archive.ubuntu.com/ubuntu noble-updates/multiverse Translation-en [7044 B]\nGet:29 http://archive.ubuntu.com/ubuntu noble-backports/main amd64 Packages [40.4 kB]\nGet:30 http://archive.ubuntu.com/ubuntu noble-backports/main Translation-en [9208 B]\nGet:31 http://archive.ubuntu.com/ubuntu noble-backports/universe amd64 Packages [30.7 kB]\nGet:32 http://archive.ubuntu.com/ubuntu noble-backports/universe Translation-en [18.2 kB]\nGet:33 http://archive.ubuntu.com/ubuntu noble-backports/multiverse amd64 Packages [780 B]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nIgn:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\nErr:34 http://archive.ubuntu.com/ubuntu noble-backports/multiverse Translation-en\n 403 Forbidden [IP: 10.148.181.1 9988]\nFetched 38.1 MB in 26s (1454 kB/s)\nReading package lists...\n'
2026-03-18 14:01:05.262 * Command standard error output: b'E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/noble-backports/multiverse/i18n/Translation-en 403 Forbidden [IP: 10.148.181.1 9988]\nE: Some index files failed to download. They have been ignored, or old ones used instead.\n'
2026-03-18 14:01:05.262 Full execution log: '/root/.local/state/testcraft/log/testcraft-20260318-135949.948147.log'
-----
.
```
Evaluation history
No evaluation history available.