### Bug Description When building in CI using the snapcraft multiarch action, `core20` builds for armhf produce a snap that fails to upload to the store. The reason is the date in the manifest.yaml (`snapcraft-started-at`) as claimed by the error: ``` - invalid-field: Unable to parse date string '2025-04-16T18:06:37.811061+00:00Z' ``` This seems to happen deterministically as every build in the last few days in our CI has [failed this way (link)](https://github.com/canonical/checkbox/actions/runs/14499410416) Here is the invalid snap (one of them): [frontend_snaps_uc20_armhf.snap.zip](https://github.com/user-attachments/files/19789999/frontend_snaps_uc20_armhf.snap.zip) We build several other snaps using the same action and all the non-core20 armhf ones are valid and correctly uploaded to the store I have tried to manually modify the manifest but the upload fails with another error (see below) **UPDATE:** unsquashing the snap, editing the date to remove the `+00:00`, quoting the field (from `snapcraft-started-at: a` to `snapcraft-started-at: 'a'`) and running `snapcraft pack` again seems to "fix" the issue **UPDATE:** from another repo using the action, this seems to affect any core20 snap (also amd64 and arm64) ### To Reproduce (I can't confirm right now if this always happens on any core20 armhf build or only in CI) ```shell git clone https://github.com/canonical/checkbox cd checkbox/checkbox-snap ./prepare_classic.sh series_classic20/ cd series_classic20/ snapcraft ``` ### Environment - Built using [snapcraft-multiarch in our CI](https://github.com/canonical/checkbox/blob/main/.github/workflows/checkbox-daily-cross-builds.yaml#L138) - Ubuntu Noble - QEMU + armhf - no snapcraft args (just `snapcraft`) ### snapcraft.yaml ```yaml name: checkbox summary: Checkbox test runner description: | Checkbox is a flexible test automation software. It’s the main tool used in Ubuntu Certification program. adopt-info: version-calculator grade: stable confinement: classic base: core20 apps: checkbox-cli: command-chain: [bin/wrapper_local] command: bin/checkbox-cli-wrapper test-runner: command-chain: [bin/wrapper_local] command: bin/test-runner sru: command-chain: [bin/wrapper_local] command: bin/sru client-cert-iot-server: command-chain: [bin/wrapper_local] command: bin/client-cert-iot-server odm-certification: command-chain: [bin/wrapper_local] command: bin/odm-certification-app configure: command: bin/configure agent: command-chain: [bin/wrapper_local] command: bin/checkbox-cli-wrapper run-agent daemon: simple restart-condition: always restart-delay: 1s shell: command-chain: [bin/wrapper_local] command: bin/shell-wrapper passthrough: hooks: configure: command-chain: [bin/wrapper_local] parts: version-calculator: plugin: dump source: . override-pull: | snapcraftctl pull # version.txt created by prepare.sh export version=`cat $SNAPCRAFT_PART_SRC/version.txt` [ $version ] || exit 1 snapcraftctl set-version $version stage: - version.txt launchers: plugin: dump source: launchers/ organize: '*': bin/ config-variables: plugin: dump source: . stage: [config_vars] workaround-automated-review-issue: plugin: nil override-build: | # This is a workaround for an upstream bug in the automated snap review tool # remove this once this issue is resolved: # https://bugs.launchpad.net/review-tools/+bug/2049093 # Note: this cats /dev/urandom because the squashfs will collapse any # predictable output to less than the desired size making this # ineffective echo "This file is a workaround for a bug in the automated snap review tool" > $SNAPCRAFT_PART_INSTALL/size_workaround echo "this only contains random bytes to pad the snap to 16kb" >> $SNAPCRAFT_PART_INSTALL/size_workaround echo "see: https://bugs.launchpad.net/review-tools/+bug/2049093" >> $SNAPCRAFT_PART_INSTALL/size_workaround head -c 16384 /dev/urandom >> $SNAPCRAFT_PART_INSTALL/size_workaround ``` ### Relevant log output ```shell Failure with the original snap: 2025-04-17 09:03:10.780 Store operation failed: 2025-04-17 09:03:10.780 - invalid-field: Unable to parse date string '2025-04-16T18:06:37.811061+00:00Z' 2025-04-17 09:03:10.781 Traceback (most recent call last): 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/application.py", line 203, in _run_inner 2025-04-17 09:03:10.781 return_code = super()._run_inner() 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_application/application.py", line 671, in _run_inner 2025-04-17 09:03:10.781 return_code = dispatcher.run() or os.EX_OK 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_cli/dispatcher.py", line 528, in run 2025-04-17 09:03:10.781 return self._loaded_command.run(self._parsed_command_args) 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/commands/upload.py", line 141, in run 2025-04-17 09:03:10.781 revision = client.notify_upload( 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/store/client.py", line 456, in notify_upload 2025-04-17 09:03:10.781 response = self.request( 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/store/client.py", line 248, in request 2025-04-17 09:03:10.781 return self.store_client.request(*args, **kwargs) 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_store/base_client.py", line 189, in request 2025-04-17 09:03:10.781 return self.http_client.request( 2025-04-17 09:03:10.781 ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 09:03:10.781 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_store/http_client.py", line 165, in request 2025-04-17 09:03:10.781 raise errors.StoreServerError(response) 2025-04-17 09:03:10.781 craft_store.errors.StoreServerError: Store operation failed: 2025-04-17 09:03:10.781 - invalid-field: Unable to parse date string '2025-04-16T18:06:37.811061+00:00Z' Failure with the modified snap: 2025-04-17 08:59:19.425 snapcraft internal error: TypeError('Object of type datetime is not JSON serializable') 2025-04-17 08:59:19.428 Traceback (most recent call last): 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_application/application.py", line 691, in run 2025-04-17 08:59:19.428 return_code = self._run_inner() 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/application.py", line 203, in _run_inner 2025-04-17 08:59:19.428 return_code = super()._run_inner() 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_application/application.py", line 671, in _run_inner 2025-04-17 08:59:19.428 return_code = dispatcher.run() or os.EX_OK 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_cli/dispatcher.py", line 528, in run 2025-04-17 08:59:19.428 return self._loaded_command.run(self._parsed_command_args) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/commands/upload.py", line 141, in run 2025-04-17 08:59:19.428 revision = client.notify_upload( 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/store/client.py", line 456, in notify_upload 2025-04-17 08:59:19.428 response = self.request( 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/snapcraft/store/client.py", line 248, in request 2025-04-17 08:59:19.428 return self.store_client.request(*args, **kwargs) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_store/base_client.py", line 189, in request 2025-04-17 08:59:19.428 return self.http_client.request( 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/craft_store/http_client.py", line 155, in request 2025-04-17 08:59:19.428 response = self._session.request( 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/requests/sessions.py", line 575, in request 2025-04-17 08:59:19.428 prep = self.prepare_request(req) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/requests/sessions.py", line 484, in prepare_request 2025-04-17 08:59:19.428 p.prepare( 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/requests/models.py", line 370, in prepare 2025-04-17 08:59:19.428 self.prepare_body(data, files, json) 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/requests/models.py", line 510, in prepare_body 2025-04-17 08:59:19.428 body = complexjson.dumps(json, allow_nan=False) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/simplejson/__init__.py", line 378, in dumps 2025-04-17 08:59:19.428 return _default_encoder.encode(obj) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/simplejson/encoder.py", line 298, in encode 2025-04-17 08:59:19.428 chunks = self.iterencode(o) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/simplejson/encoder.py", line 379, in iterencode 2025-04-17 08:59:19.428 return _iterencode(o, 0) 2025-04-17 08:59:19.428 ^^^^^^^^^^^^^^^^^ 2025-04-17 08:59:19.428 File "/var/lib/snapd/snap/snapcraft/14167/lib/python3.12/site-packages/simplejson/encoder.py", line 274, in default 2025-04-17 08:59:19.428 raise TypeError('Object of type %s is not JSON serializable' % 2025-04-17 08:59:19.428 TypeError: Object of type datetime is not JSON serializable ``` ### Additional context Full build command in the action: https://github.com/canonical/snapcraft-multiarch-action/blob/master/src/build.ts#L122 Action invocation: https://github.com/canonical/checkbox/blob/main/.github/workflows/checkbox-daily-cross-builds.yaml#L59