← Back to issue list

tests: store resources spread test is flaky

View original Github issue

Metadata

Project
charmcraft
Number
#1218
Type
issue
State
closed
Author
lengau
Labels
—
Created
Updated
Closed

Current evaluation

Flaky store resources spread test was fixed by merged PR canonical/charmcraft#1223 in commit 35993996.

Resolution: closed resolved

Reason: PR #1223 was merged as commit 35993996 with a two-file fix to the resource spread test, directly addressing the reported status-polling timeout.

Evaluated by: mr-cal-bot

No scores available.

Issue body

### Bug Description The spread test at `tests/spread/store/resources` is flaky and appears to be breaking more frequently lately. ### To Reproduce `spread multipass:tests/spread/store/resources` Or look at CI: https://github.com/canonical/charmcraft/actions/runs/5880633896/job/15947762402?pr=1206 ### Environment spread :-) ### charmcraft.yaml ```shell n/a ``` ### Relevant log output ```shell Creating the package itself Created 'gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm'. Charm packed ok Charms packed: gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm ++ jq .revision +++ ls gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm ++ charmcraft upload gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm --format=json Starting to push 'gh-ci-charmcraft-charm_ubuntu-22.04-amd64.charm' Uploading... (--->) Uploading... (<---) Uploading bytes ended, id 6f549bc1-675b-497d-a58c-24904b63a0d1 Upload 6f549bc1-675b-497d-a58c-24904b63a0d1 started, got status url /v1/charm/gh-ci-charmcraft-charm/revisions/review?upload-id=6f549bc1-675b-497d-a58c-24904b63a0d1 Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '6f549bc1-675b-497d-a58c-24904b63a0d1'}]} Status checked: {'revisions': [{'errors': None, 'revision': 103, 'status': 'scanned', 'upload-id': '6f549bc1-675b-497d-a58c-24904b63a0d1'}]} Status checked: {'revisions': [{'errors': None, 'revision': 103, 'status': 'approved', 'upload-id': '6f549bc1-675b-497d-a58c-24904b63a0d1'}]} + last_charm_revno=103 ++ paste -sd, - ++ sort ++ jq -r --arg revno 103 '.[] | select(.charm_revision|tostring==$revno) | .name' ++ charmcraft resources gh-ci-charmcraft-charm --format=json + last_charm_resources=example-file,example-image + test example-file,example-image = example-file,example-image + charmcraft upload-resource gh-ci-charmcraft-charm example-file --file=resourcefile.txt Uploading resource directly from file 'resourcefile.txt'. Starting to push 'resourcefile.txt' Uploading... (--->) Uploading... (<---) Uploading bytes ended, id 316911ae-123f-45db-8854-7174d4b57840 Upload 316911ae-123f-45db-8854-7174d4b57840 started, got status url /v1/charm/gh-ci-charmcraft-charm/revisions/review?upload-id=316911ae-123f-45db-8854-7174d4b57840 Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Status checked: {'revisions': [{'errors': None, 'revision': None, 'status': 'new', 'upload-id': '316911ae-123f-45db-8854-7174d4b57840'}]} Timeout polling Charmhub for upload status (after 60.0s). Full execution log: '/root/.local/state/charmcraft/log/charmcraft-20230816-153359.336875.log' ```

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed resolved Flaky store resources spread test was fixed by merged PR canonical/charmcraft#1223 in commit 35993996.
stealth/space-bunny-alpha — closed resolved Flaky resource spread test fixed by merged PR canonical/charmcraft#1223 (commit 35993996).
stealth/space-bunny-alpha — closed resolved Fixed by merged PR #1223 (commit 35993996), which updated the resources spread test to make it more robust.
qwen3.6-35b-a3b-mtp-q6 — — The flaky store resources spread test was closed without comments or a fix, likely abandoned or auto-closed. The Charmhub upload timeout issue remains unresolved.
qwen/qwen3.6-35b-a3b — — Closed without comments or linked fixes. The flaky spread test timeout issue was abandoned or resolved by infrastructure changes.
qwen/qwen3.6-35b-a3b — — Closed without comments or linked resolution. The flaky store resources spread test timeout was reported but never addressed or fixed, indicating abandonment.
qwen/qwen3.6-35b-a3b — — Closed without resolution after nine days, likely due to inactivity. The flaky tests/spread/store/resources test, which timed out polling Charmhub for upload status, remains unaddressed.
qwen3.6-35b-a3b-mtp-q6 — — Closed without comments. The flaky tests/spread/store/resources test timed out polling Charmhub during uploads, indicating it was abandoned or resolved without tracking.

Update history

No update history recorded yet.

Related work

Related issues

Issue Project State Summary Similarity
#1223 tests: fix resource spread test charmcraft merged Merged #1223 (commit 35993996), fixing the flaky store-resources spread test and advancing #1193; related store fixes landed in #1226 and #1234.
82%
#1469 Charm upload/release spread test is flaky charmcraft closed Fixed the flaky store spread test by waiting and retrying release checks; merged in canonical/charmcraft#1460 (fd5bd6ba).
81%
#1712 Store resources spread test is broken charmcraft closed CI's store-resources spread test returned an empty value; PR #1761 temporarily disabled the related upload-resource test, but no permanent fix was recorded.
77%
#1429 spread: charm upload and release store test is flaky when the store is busy charmcraft closed Fixed in merged PR #1460 (commit fd5bd6ba), which reduced timing sensitivity in store upload and release Spread tests.
76%
#1460 fix(tests): make store spread tests less flaky charmcraft merged Merged PR #1460 as fd5bd6ba, making store spread tests less timing-sensitive and fixing flaky upload/release checks in #1429.
74%
#1214 tests: add more charms to large spread tests charmcraft closed Closed unmerged after 1,135 days; its large-spread test expansion landed through merged PR canonical/charmcraft#1217 in commit 0f4591a9.
73%
#1736 fix: fix some edge cases with resource uploads charmcraft merged Merged PR #1736 as commit 97603da4, fixing resource-upload edge cases and resolving charmcraft#1712's empty-resource spread-test failure.
72%
#5954 tests: fix store tests snapcraft merged PR #5954 merged as commit 0be1e384d: store spread tests switched to new staging credentials/key test-staging-key-2 and to wheel-based automated tests (as in #5885).
72%
#4411 Store spread tests failing snapcraft closed The store spread-test authentication failure was confirmed fixed by maintainer @mr-cal on 7 May 2024; no resolving PR or commit was referenced.
71%
#1427 tests(spread): make dependencies restore work even if no charm is built charmcraft merged Merged in canonical/charmcraft#1427 (commit 2d432044), making spread-test dependency restoration work when no charm artifact is built.
71%