Charmhub integration tests are getting skipped in CI
Metadata
Current evaluation
Charmhub integration tests are skipped in CI because CRAFT_STORE_CHARMCRAFT_CREDENTIALS is not set in the QA workflow. Triaged and tracked in Jira (CRAFT-4676) but no fix has been merged.
Suggested action: keep open
Reason: The issue is triaged (Status: Triaged label) and tracked in Jira (CRAFT-4676). The QA workflow currently passes STAGING_SSO_USERNAME/PASSWORD but does not pass CRAFT_STORE_CHARMCRAFT_CREDENTIALS, which is why the charmhub tests are skipped. The fix is straightforward (add the secret to the workflow) but requires a valid staging credential to be configured in the repo's GitHub secrets, which is an operational step outside the codebase. No fix commit has been found in the git history.
Impact:
50
Quick Win:
35.0
Staleness:
55
Complexity:
30
Confidence:
70
Support Request:
0
Issue body
### Bug Description
CI is skipping the integration tests that push to charmhub.
### To Reproduce
https://github.com/canonical/craft-store/actions/runs/15936558372/job/44957531647
### part yaml
```shell
```
### Relevant log output
```shell
============================= test session starts ==============================
collecting ... collected 282 items / 254 deselected / 28 selected
tests/integration/test_get_list_releases.py::test_charm_get_list_releases SKIPPED [ 3%]
tests/integration/test_update_resource_revisions.py::test_charm_update_resource_revisions SKIPPED [ 7%]
tests/integration/charmhub/test_charmhub_workflow.py::test_full_charm_workflow SKIPPED [ 10%]
tests/integration/test_push_resource.py::test_charm_push_resource[None] SKIPPED [ 14%]
tests/integration/test_push_resource.py::test_charm_push_resource[bases2] SKIPPED [ 17%]
tests/integration/test_push_resource.py::test_charm_push_resource[bases3] SKIPPED [ 21%]
tests/integration/test_push_resource.py::test_charm_push_resource[bases1] SKIPPED [ 25%]
tests/integration/test_upload.py::test_charm_upload SKIPPED (CRAFT_S...) [ 28%]
tests/integration/test_release.py::test_charm_release SKIPPED (CRAFT...) [ 32%]
tests/integration/publisher/test_read.py::test_list_releases SKIPPED [ 35%]
tests/integration/publisher/test_read.py::test_get_package_metadata SKIPPED [ 39%]
tests/integration/publisher/test_read.py::test_list_revisions SKIPPED [ 42%]
tests/integration/test_list_resource_revisions.py::test_charm_list_resource_revisions SKIPPED [ 46%]
tests/integration/test_list_revisions.py::test_charm_list_revisions SKIPPED [ 50%]
tests/integration/test_register_unregister.py::test_register_unregister_cycle[bundle] SKIPPED [ 53%]
tests/integration/test_register_unregister.py::test_unregister_nonexistent SKIPPED [ 57%]
tests/integration/test_register_unregister.py::test_register_unregister_cycle[charm] SKIPPED [ 60%]
tests/integration/publisher/test_write.py::test_register_unregister_cycle[charm] SKIPPED [ 64%]
tests/integration/publisher/test_write.py::test_release SKIPPED (CRA...) [ 67%]
tests/integration/publisher/test_write.py::test_create_tracks[0.32-None] SKIPPED [ 71%]
tests/integration/publisher/test_write.py::test_create_tracks[None-None] SKIPPED [ 75%]
tests/integration/publisher/test_write.py::test_register_unregister_cycle[bundle] SKIPPED [ 78%]
tests/integration/publisher/test_write.py::test_create_tracks[0.32-\\d+] SKIPPED [ 82%]
tests/integration/publisher/test_write.py::test_create_tracks[None-\\d+] SKIPPED [ 85%]
tests/integration/publisher/test_write.py::test_create_tracks[50-\\d+] SKIPPED [ 89%]
tests/integration/publisher/test_write.py::test_create_tracks[50-None] SKIPPED [ 92%]
tests/integration/publisher/test_write.py::test_create_disallowed_track SKIPPED [ 96%]
tests/integration/publisher/test_write.py::test_create_existing_track SKIPPED [100%]
===================== 28 skipped, 254 deselected in 0.94s ======================
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.8-27b |
Impact:
50
Quick Win:
35.0
Staleness:
55
Complexity:
30
Confidence:
70
Support Request:
0
|
keep open | Charmhub integration tests are skipped in CI because CRAFT_STORE_CHARMCRAFT_CREDENTIALS is not set in the QA workflow. Triaged and tracked in Jira (CRAFT-4676) but no fix has been merged. | |
| qwen/qwen3.6-35b-a3b |
Staleness:
92
Complexity:
20
Confidence:
75
Support Request:
10
|
close stale | Integration tests for Charmhub are consistently skipped in CI due to missing environment variables or configuration. The issue is triaged but has seen no maintainer activity for over a year. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
90
Complexity:
30
Confidence:
75
Support Request:
10
|
keep open | Integration tests for Charmhub are consistently skipped in CI due to missing environment variables or configuration. The issue is triaged but has been inactive for over a year with no maintainer follow-up or fix. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
90
Complexity:
25
Confidence:
75
Support Request:
5
|
keep open | Charmhub integration tests are skipped in CI due to missing environment variables. Triaged but inactive for over ten months, awaiting assignment or investigation to restore test execution. |
Update history
No update history recorded yet.
Related work
-
Related To:
craft-store#393
(confidence 75%)
PR #393 added STAGING_SSO_USERNAME/PASSWORD to the QA workflow but did not add CRAFT_STORE_CHARMCRAFT_CREDENTIALS, which is the env var the charmhub tests require.
Related issues
No related issues found above the similarity threshold.