Snap store endpoints test is flaky

Metadata

Project: craft-store
Number: #243
Type: issue
State: open
Author: lengau
Labels: —
Created: 2024-12-11 11:47 PM UTC
Updated: 2025-05-01 04:43 PM UTC
Closed: —

Current evaluation

Flaky unit test tests/unit/test_endpoints.py::test_snap_store: expires timestamp differs by one second under load. Root-caused to the `expires` fixture; fix merged in PR #391 (c42cd69) but the issue was never closed.

Suggested action: close resolved

Reason: The root cause is the `expires` fixture in tests/unit/conftest.py: the expected value is built as `(now + timedelta(seconds=seconds)).replace(microsecond=0)` while craft_store/endpoints.py builds `(datetime.now(tz=timezone.utc).replace(microsecond=0) + timedelta(seconds=ttl)).isoformat()`, so a sub-second rounding difference under CI load produces the '00:00:22' vs '00:00:23' diff shown in the report. The most recent commit touching tests/unit/conftest.py and tests/unit/test_endpoints.py is c42cd69 "fix(test): make endpoint tests less flaky (#391)", a merged PR that adjusts those two test files to reduce endpoint-test flakiness in slow CI environments — i.e. the fix is already in the tree and the issue is only unclosed housekeeping. Impact is limited to CI noise (no library behaviour change), and the only comment is an autogenerated Jira-sync bot message, so no maintainer triage discussion exists.

Evaluated by: mr-cal-bot

Impact: 32 Quick Win: 22 Complexity: 18 Confidence: 86 Actionability: 85

Issue body

### Bug Description `tests/unit/test_endpoints.py::test_snap_store` is flaky, as the expiry time can be off by a second due to the way it is generated. ### To Reproduce Put your machine under load, then run: `uv run pytest tests/unit/test_endpoints.py::test_snap_store` until you get it. Or see: https://github.com/canonical/craft-store/actions/runs/12286865000/job/34287817294?pr=241 ### part yaml ```shell n/a ``` ### Relevant log output ```shell ================================== FAILURES =================================== _______________________________ test_snap_store _______________________________ expires = <function expires.<locals>.offset_iso_dt at 0x000002D4A62ED800> def test_snap_store(expires): snap_store = endpoints.SNAP_STORE assert snap_store.tokens == "/api/v2/tokens" assert snap_store.tokens_exchange == "/api/v2/tokens/exchange" assert snap_store.whoami == "/api/v2/tokens/whoami" assert snap_store.upload == "/unscanned-upload/" > assert snap_store.get_token_request( permissions=["permission-foo", "permission-bar"], description="client description", ttl=1000, ) == { "permissions": ["permission-foo", "permission-bar"], "description": "client description", "expires": expires(1000), } E AssertionError: assert {'description...mission-bar']} == {'description...mission-bar']} E E Omitting 2 identical items, use -vv to show E Differing items: E {'expires': '2024-12-12T00:00:23+00:00'} != {'expires': '2024-12-12T00:00:22+00:00'} E E Full diff: E { E 'description': 'client description', E - 'expires': '2024-12-12T00:00:22+00:00', E ? ^ E + 'expires': '2024-12-12T00:00:23+00:00', E ? ^ E 'permissions': [ E 'permission-foo', E 'permission-bar', E ], E } tests\unit\test_endpoints.py:119: AssertionError =========================== short test summary info =========================== FAILED tests/unit/test_endpoints.py::test_snap_store - AssertionError: assert {'description...mission-bar']} == {'description...mission-bar']} Omitting 2 identical items, use -vv to show Differing items: {'expires': '2024-12-12T00:00:23+00:00'} != {'expires': '2024-12-12T00:00:22+00:00'} Full diff: { 'description': 'client description', - 'expires': '2024-12-12T00:00:22+00:00', ? ^ + 'expires': '2024-12-12T00:00:23+00:00', ? ^ 'permissions': [ 'permission-foo', 'permission-bar', ], } ================ 1 failed, 200 passed, 14 deselected in 4.78s ================= ```

Evaluation history

Date	Model	Scores	Action	Summary
2026-10-02 05:56 PM UTC	stealth/space-bunny-alpha	Impact: 32 Quick Win: 22 Complexity: 18 Confidence: 86 Actionability: 85	close resolved	Flaky unit test tests/unit/test_endpoints.py::test_snap_store: expires timestamp differs by one second under load. Root-caused to the `expires` fixture; fix merged in PR #391 (c42cd69) but the issue was never closed.
2026-09-21 07:20 PM UTC	qwen/qwen3.6-35b-a3b	Impact: 20 Quick Win: 14 Complexity: 10 Confidence: 75 Actionability: 80	close resolved	Flaky test_snap_store test due to expires timestamp off by one second; fix merged in PR #391 but issue remains open
2026-09-01 05:13 AM UTC	qwen/qwen3.8-27b	Impact: 25 Quick Win: 21 Staleness: 75 Complexity: 15 Confidence: 65 Support Request: 0	needs triage	Flaky unit test test_snap_store: token expiry timestamp can be off by one second under load, causing intermittent CI failures. Unlabeled, unassigned, no maintainer response since filing ~20 months ago; only a Jira-sync bot comment.
2026-07-22 12:17 PM UTC	qwen/qwen3.6-35b-a3b	Staleness: 95 Complexity: 20 Confidence: 85 Support Request: 5	needs triage	Flaky unit test for snap store endpoints where the generated token expiry timestamp differs by one second. Unassigned, unlabelled, and inactive for over a year.
2026-07-09 03:58 AM UTC	qwen3.6-35b-a3b-mtp-q6	Staleness: 95 Complexity: 25 Confidence: 85 Support Request: 5	close stale	Flaky unit test test_snap_store fails due to a 1-second discrepancy in the generated token expiry timestamp. Open for over 1.5 years with no maintainer response or fix.

Update history

No update history recorded yet.

Related work

Likely Fixed By: craft-store#391 (confidence 93%)
Merged PR 'fix(test): make endpoint tests less flaky' (commit c42cd69) is the newest commit on tests/unit/conftest.py and tests/unit/test_endpoints.py and adjusts those tests for slow CI.
Related To: craft-store#241 (confidence 90%)
The issue body links the failing Actions job (runs/12286865000/job/34287817294?pr=241) from PR #241 'chore: move tests to starflow', the run where the flake was observed.

Related issues

No related issues found above the similarity threshold.