Charmhub integration tests are getting skipped in CI
Metadata
Current evaluation
28 Charmhub integration tests remain skipped because CI supplies SSO credentials but the tests gate on unset CRAFT_STORE_CHARMCRAFT_CREDENTIALS.
Suggested action: keep open
Reason: The current QA workflow passes only STAGING_SSO_USERNAME and STAGING_SSO_PASSWORD, while tests/integration/conftest.py skips Charmhub tests when CRAFT_STORE_CHARMCRAFT_CREDENTIALS is absent. This triaged bug has a precise failing run and confirmed configuration mismatch, but needs a defined CI authentication bridge.
Evaluated by: mr-cal-bot
Impact:
48
Quick Win:
18
Complexity:
52
Confidence:
93
Actionability:
78
Issue body
### Bug Description
CI is skipping the integration tests that push to charmhub.
### To Reproduce
https://github.com/canonical/craft-store/actions/runs/15936558372/job/44957531647
### part yaml
```shell
```
### Relevant log output
```shell
============================= test session starts ==============================
collecting ... collected 282 items / 254 deselected / 28 selected
tests/integration/test_get_list_releases.py::test_charm_get_list_releases SKIPPED [ 3%]
tests/integration/test_update_resource_revisions.py::test_charm_update_resource_revisions SKIPPED [ 7%]
tests/integration/charmhub/test_charmhub_workflow.py::test_full_charm_workflow SKIPPED [ 10%]
tests/integration/test_push_resource.py::test_charm_push_resource[None] SKIPPED [ 14%]
tests/integration/test_push_resource.py::test_charm_push_resource[bases2] SKIPPED [ 17%]
tests/integration/test_push_resource.py::test_charm_push_resource[bases3] SKIPPED [ 21%]
tests/integration/test_push_resource.py::test_charm_push_resource[bases1] SKIPPED [ 25%]
tests/integration/test_upload.py::test_charm_upload SKIPPED (CRAFT_S...) [ 28%]
tests/integration/test_release.py::test_charm_release SKIPPED (CRAFT...) [ 32%]
tests/integration/publisher/test_read.py::test_list_releases SKIPPED [ 35%]
tests/integration/publisher/test_read.py::test_get_package_metadata SKIPPED [ 39%]
tests/integration/publisher/test_read.py::test_list_revisions SKIPPED [ 42%]
tests/integration/test_list_resource_revisions.py::test_charm_list_resource_revisions SKIPPED [ 46%]
tests/integration/test_list_revisions.py::test_charm_list_revisions SKIPPED [ 50%]
tests/integration/test_register_unregister.py::test_register_unregister_cycle[bundle] SKIPPED [ 53%]
tests/integration/test_register_unregister.py::test_unregister_nonexistent SKIPPED [ 57%]
tests/integration/test_register_unregister.py::test_register_unregister_cycle[charm] SKIPPED [ 60%]
tests/integration/publisher/test_write.py::test_register_unregister_cycle[charm] SKIPPED [ 64%]
tests/integration/publisher/test_write.py::test_release SKIPPED (CRA...) [ 67%]
tests/integration/publisher/test_write.py::test_create_tracks[0.32-None] SKIPPED [ 71%]
tests/integration/publisher/test_write.py::test_create_tracks[None-None] SKIPPED [ 75%]
tests/integration/publisher/test_write.py::test_register_unregister_cycle[bundle] SKIPPED [ 78%]
tests/integration/publisher/test_write.py::test_create_tracks[0.32-\\d+] SKIPPED [ 82%]
tests/integration/publisher/test_write.py::test_create_tracks[None-\\d+] SKIPPED [ 85%]
tests/integration/publisher/test_write.py::test_create_tracks[50-\\d+] SKIPPED [ 89%]
tests/integration/publisher/test_write.py::test_create_tracks[50-None] SKIPPED [ 92%]
tests/integration/publisher/test_write.py::test_create_disallowed_track SKIPPED [ 96%]
tests/integration/publisher/test_write.py::test_create_existing_track SKIPPED [100%]
===================== 28 skipped, 254 deselected in 0.94s ======================
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha |
Impact:
48
Quick Win:
18
Complexity:
52
Confidence:
93
Actionability:
78
|
keep open | 28 Charmhub integration tests remain skipped because CI supplies SSO credentials but the tests gate on unset CRAFT_STORE_CHARMCRAFT_CREDENTIALS. | |
| qwen/qwen3.6-35b-a3b |
Impact:
70
Quick Win:
39
Complexity:
30
Confidence:
70
Actionability:
80
|
keep open | Charmhub integration tests are skipped in CI because CRAFT_STORE_CHARMCRAFT_CREDENTIALS is not set, preventing test execution | |
| qwen/qwen3.8-27b |
Impact:
50
Quick Win:
35
Staleness:
55
Complexity:
30
Confidence:
70
Support Request:
0
|
keep open | Charmhub integration tests are skipped in CI because CRAFT_STORE_CHARMCRAFT_CREDENTIALS is not set in the QA workflow. Triaged and tracked in Jira (CRAFT-4676) but no fix has been merged. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
92
Complexity:
20
Confidence:
75
Support Request:
10
|
close stale | Integration tests for Charmhub are consistently skipped in CI due to missing environment variables or configuration. The issue is triaged but has seen no maintainer activity for over a year. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
90
Complexity:
30
Confidence:
75
Support Request:
10
|
keep open | Integration tests for Charmhub are consistently skipped in CI due to missing environment variables or configuration. The issue is triaged but has been inactive for over a year with no maintainer follow-up or fix. |
Update history
No update history recorded yet.
Related work
-
Related To:
warthogs#4676
(confidence 100%)
The bot comment reports that internal ticket CRAFT-4676 was created; no resolution is reported.
Related issues
No related issues found above the similarity threshold.