Have a way for charmcraft to run the tests
Metadata
Current evaluation
No dedicated test command was added; maintainers directed users to override-build in charmcraft.yaml, running craftctl build before tests such as pytest.
Resolution: closed not a bug
Reason: Closed after a maintainer comment on 2022-03-17 showed that tests can already run through override-build using craftctl build followed by pytest, making special test machinery unnecessary.
Evaluated by: mr-cal-bot
No scores available.
Issue body
Charm tests needs to be run by charmcraft. We can add a `test` command, for that.
This is because the charm building can be complex, and will be specified in `charmcraft.yaml`, so no need to duplicate that specification (and the environment building) outside of charmcraft.
A wonderful side benefit of this is that will be trivial to run the tests in all the supported platforms.
That said, we need for `charmcraft test` by default to be super fast (without any noticeable overhead), otherwise developing a charm will be a pain in the neck. We probably should run the tests in only one platform by default (maybe in the newest of the specified ones), but even so...
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed not a bug | No dedicated test command was added; maintainers directed users to override-build in charmcraft.yaml, running craftctl build before tests such as pytest. | |
| stealth/space-bunny-alpha | — | closed not a bug | Closed as an unnecessary feature request; the maintainer recommended running tests through charmcraft.yaml override-build instead of adding a dedicated test command. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Rejected. The maintainer closed the issue, stating a dedicated test command is unnecessary. Users can run tests by configuring override-build in charmcraft.yaml to execute pytest after the build step. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed as unnecessary. The maintainer confirmed that tests can already be executed during the build process using the override-build directive in charmcraft.yaml, making a dedicated test command redundant. | |
| qwen/qwen3.6-35b-a3b | — | — | The proposed test command was rejected. The maintainer closed the issue, noting that tests can already be executed using the existing override-build directive in charmcraft.yaml, rendering additional machinery unnecessary. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Rejected. The proposed test command and pre-pack-hook were deemed unnecessary. Users can already run tests during the build process by appending commands to the override-build directive in charmcraft.yaml, making special machinery redundant. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1092 Some tests will require the charm plugin to be enabled | craft-application | open | Open, unlabelled issue: charmcraft PR #2722 may make craft-application tests depend on the charm plugin; the maintainer asks to remove that dependency, but no affected test or fix is identified. | |
| #1851 How-to doc for charmcraft test | charmcraft | open | Documentation request from a maintainer to write a how-to guide for the undocumented `charmcraft test` command. Open with no labels, assignee, or maintainer triage; only an automated Jira-sync bot comment (CRAFT-3957). |