spread: charm upload and release store test is flaky when the store is busy
Metadata
Current evaluation
Fixed in merged PR #1460 (commit fd5bd6ba), which reduced timing sensitivity in store upload and release Spread tests.
Resolution: closed resolved
Reason: PR #1460 was merged as commit fd5bd6ba and made the store Spread tests less flaky under slow upload and release processing.
Evaluated by: mr-cal-bot
No scores available.
Issue body
### Bug Description
The `charm-upload-and-release` store test sometimes fails because the store sometimes takes longer than expected to process a charm.
### To Reproduce
Run the snap-store-tests action when the staging store has a lot of chramcraft-related activity.
### Environment
GitHub CI: https://github.com/canonical/charmcraft/actions/runs/7130696852/job/19418225138#step:6:229
### charmcraft.yaml
```shell
n/a
```
### Relevant log output
```shell
++ charmcraft upload gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm --format=json
Starting to push 'gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm'
Uploading... (--->)
Uploading... (<---)
Uploading bytes ended, id 3b7b89b0-e66c-4fce-a951-bd8e783aadb4
Upload 3b7b89b0-e66c-4fce-a951-bd8e783aadb4 started, got status url /v1/charm/gh-ci-charmcraft-charm/revisions/review?upload-id=3b7b89b0-e66c-4fce-a951-bd8e783aadb4
Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '3b7b89b0-e66c-4fce-a951-bd8e783aadb4'}]}
Status checked: {'revisions': [{'errors': None, 'revision': 1345, 'status': 'new', 'upload-id': '3b7b89b0-e66c-4fce-a951-bd8e783aadb4'}]}
Status checked: {'revisions': [{'errors': None, 'revision': 1345, 'status': 'approved', 'upload-id': '3b7b89b0-e66c-4fce-a951-bd8e783aadb4'}]}
+ uploaded_revno=1345
++ jq -r --arg revno 1345 '.[] | select(.revision|tostring==$revno)'
++ charmcraft revisions gh-ci-charmcraft-charm --format=json
+ up_revision='{
"revision": 1345,
"version": "1345",
"created_at": "2023-12-07T16:18:25Z",
"status": "approved"
}'
++ jq -r .created_at
++ echo '{' '"revision":' 1345, '"version":' '"1345",' '"created_at":' '"2023-12-07T16:18:25Z",' '"status":' '"approved"' '}'
+ up_revision_created=2023-12-07T16:18:25Z
+ [[ 2023-12-07T16:15:37+00:00 < 2023-12-07T16:18:25Z ]]
+ charmcraft release gh-ci-charmcraft-charm -r 1345 -c edge
Revision 1345 of 'gh-ci-charmcraft-charm' released to edge
++ jq -r '.[] | select(.track=="latest") | .mappings[0].releases | .[] | select(.channel=="latest/edge")'
++ charmcraft status gh-ci-charmcraft-charm --format=json
+ edge_release='{
"status": "open",
"channel": "latest/edge",
"version": "1344",
"revision": 1344,
"resources": [
{
"name": "example-file",
"revision": 1
},
{
"name": "example-image",
"revision": 1
}
],
"expires_at": null
}'
++ jq -r .revision
++ echo '{' '"status":' '"open",' '"channel":' '"latest/edge",' '"version":' '"1344",' '"revision":' 1344, '"resources":' '[' '{' '"name":' '"example-file",' '"revision":' 1 '},' '{' '"name":' '"example-image",' '"revision":' 1 '}' '],' '"expires_at":' null '}'
+ edge_revision=1344
+ test 1344 -ge 1345
-----
.
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Fixed in merged PR #1460 (commit fd5bd6ba), which reduced timing sensitivity in store upload and release Spread tests. | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed store-test flakiness in PR #1460 (commit fd5bd6ba), making spread tests less timing-sensitive when store processing is slow. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without comments or linked resolution. The flaky store test failure due to processing delays was abandoned or resolved via CI configuration adjustments. | |
| qwen/qwen3.6-35b-a3b | — | — | Abandoned. The charm-upload-and-release test fails during high store load due to processing delays, but no fix was implemented. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without resolution or linked pull request. The flaky charm upload and release test failure during high store load remains unfixed, indicating the issue was abandoned. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without resolution. The flaky charm-upload-and-release test failed due to store processing delays causing timestamp mismatches. Zero comments and no linked fixes indicate the issue was abandoned due to inactivity. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without a fix. The flaky charm-upload-and-release test failure caused by store latency was abandoned. No resolution or code changes were applied, indicating the issue was deferred. |
Update history
No update history recorded yet.
Related work
-
Likely Fixed By:
canonical/charmcraft#1460
(confidence 100%)
Merged fix titled “fix(tests): make store spread tests less flaky”.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1469 Charm upload/release spread test is flaky | charmcraft | closed | Fixed the flaky store spread test by waiting and retrying release checks; merged in canonical/charmcraft#1460 (fd5bd6ba). | |
| #1460 fix(tests): make store spread tests less flaky | charmcraft | merged | Merged PR #1460 as fd5bd6ba, making store spread tests less timing-sensitive and fixing flaky upload/release checks in #1429. | |
| #1362 CI: `charm-upload-and-release` test is flaky | charmcraft | closed | Added propagation delays and status retries in PR #1366 (commit 05b18a03); later store-test hardening landed in PR #1460 (fd5bd6ba). | |
| #1218 tests: store resources spread test is flaky | charmcraft | closed | Flaky store resources spread test was fixed by merged PR canonical/charmcraft#1223 in commit 35993996. | |
| #1366 tests(spread): make release test less flaky | charmcraft | merged | Merged #1366 at commit 05b18a03, reducing flakiness in the Spread release test and resolving #1362. | |
| #1292 test(spread): make library tests register/unregister charms | charmcraft | merged | Merged #1292 in commit 99a5a505, making library spread tests register and unregister charms and fixing store-timeout failures in #1279. | |
| #909 (qa) Spread tests that exercise all store related operations (CRAFT-1080) | charmcraft | merged | Merged #909 in e5ac156c, adding Spread coverage for all Charmcraft store operations under CRAFT-1080 (8 files, +332/-4). | |
| #1209 Store library update: spread test not working | charmcraft | closed | Store publish-lib spread test failed because the API JSON had a different structure than jq expected. Fixed by merged PR #1213 ("tests: fix API json structure", commit e18eab3a), which corrected the test's JSON assertions. | |
| #1324 test(spread): test-deploy charms in large spread tests | charmcraft | merged | Merged PR #1324 via commit a14d6d91, adding test deployment of charms and bundles to large Spread tests. | |
| #1290 test(spread): clean up store tests | charmcraft | merged | Merged in commit 34ab4bdd, cleaning up store spread tests by deleting 28 lines across four files; the remaining library-test failure was fixed separately in #1292. |