tests: update the store failure tests
Metadata
Current evaluation
Merged after approval and passing CI. Updates store failure tests by adjusting autopkgtest scripts to report final errors, sets exception exit codes to 2, and clarifies integration test execution against fake servers.
Suggested action: —
No scores available.
Issue body
- [x] Have you followed the [guidelines for contributing](https://github.com/snapcore/snapcraft/blob/master/CONTRIBUTING.md)?
- [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [x] If this is a bugfix. Have you checked that there is a bug report open for the issue you are trying to fix on [bug reports](https://bugs.launchpad.net/snapcraft)?
- [x] If this is a new feature. Have you discussed the design on the [forum](https://forum.snapcraft.io)?
- [x] Have you successfully run `./runtests.sh static`?
- [x] Have you successfully run `./runtests.sh unit`?
-----
There is a good bunch of things going on here.
First, the autopkgtests scripts are run without -e, so only the last error is reported.
Then, through a set of changes, when there is an exception that causes snapcraft to exit the return code is 2 ( https://github.com/snapcore/snapcraft/blob/67089e05002c5fcbfed0d64ac1b3f69363102e3a/snapcraft/internal/errors.py#L35 ).
Finally, this only happens when the integration tests are run against the fake servers. Otherwise, the tests are skipped. Now this is confusing, because the pull requests from forks should run against the fakes, and should have caught this. However, the tests are being skipped, which means that TEST_STORE env var is not set to `fake`. Probably, we are not passing the right var to the container, but I will take care of that tomorrow.
I'm sorry about this list of wrongs.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Merged after approval and passing CI. Updates store failure tests by adjusting autopkgtest scripts to report final errors, sets exception exit codes to 2, and clarifies integration test execution against fake servers. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged test updates adjusting exception return codes and clarifying fake server execution. Resolves environment variable configuration for automated test runs. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged test updates for store failure scenarios. Updated autopkgtest scripts to report all errors, set exception exit codes to 2, and clarified integration test behavior with fake servers and the TEST_STORE variable. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #5954 tests: fix store tests | snapcraft | merged | Merged after approval. Updated store tests with new staging credentials and switched to wheels for automated spread tests. CI passed, with unrelated spread failures noted by the author. | |
| #86 tests: update store tests for new image | craft-store | merged | Merged after approval. Updates store tests for a new image and adds replication instructions. All tests passed successfully. | |
| #1716 tests: split the integration autopkgtests | snapcraft | merged | Merged after splitting integration autopkgtests to eliminate timeout failures. The change adds minor per-test setup overhead but resolves recurring execution timeouts. Approved by two reviewers with passing CI checks. | |
| #2903 tests: fix status test for staging store | snapcraft | merged | Merged into master after approval and passing CI. The change corrected the staging store status test with a five-line update across one file, resulting in negligible coverage impact. | |
| #1247 Fixing Store integration tests. | snapcraft | merged | Merged updates to Store integration tests accommodating a new 40-character snap-name size limit. Test prefixes and configurations were adjusted to pass CI, with further error handling improvements planned. | |
| #1984 tests: update store tests user | snapcraft | merged | Merged after removing outdated tests and updating the store test user and prefix. Added documentation and warnings. Approved by @thomir. Changes span 13 files, reducing code by 73 lines. |