← Back to issue list

[WIP] tests: use pytest as test runner

View original Github issue

Metadata

Project
snapcraft
Number
#2176
Type
pull request
State
closed
Author
kyrofa
Labels
—
Created
Updated
Closed

Current evaluation

WIP PR to run snapcraft unit tests with pytest (+xdist) sat idle and was closed unmerged; the migration later landed in #3167 (788475f62) after #3162 dropped testscenarios.

Resolution: closed superseded

Reason: The WIP branch was never merged and was closed after ~1 year of inactivity, with the author noting it was 'difficult to move things over piecemeal'. The goal was achieved separately: #3162 (58c66b7a7) removed testscenarios from tests/unit/lifecycle/test_order.py and PR #3167 'unit tests: move to pytest' (788475f62) landed the pytest migration, adding pytest-cov/fixtures while keeping unittest for integration tests.

Evaluated by: mr-cal-bot

No scores available.

Issue body

- [ ] Have you followed the [guidelines for contributing](https://github.com/snapcore/snapcraft/blob/master/CONTRIBUTING.md)? - [ ] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)? - [ ] If this is a bugfix. Have you checked that there is a bug report open for the issue you are trying to fix on [bug reports](https://bugs.launchpad.net/snapcraft)? - [ ] If this is a new feature. Have you discussed the design on the [forum](https://forum.snapcraft.io)? - [ ] Have you successfully run `./runtests.sh static`? - [ ] Have you successfully run `./runtests.sh unit`? ----- Don't rewrite unittest classes to use pytest, but simply start using pytest as the runner, which gives us the ability to use xdist, running tests in parallel. This PR halves the unit test run time on my local machine. The savings won't be that dramatic on Travis, but it should help. Integration test time savings are unknown, but should be non-negligible as well. The other changes here are a side effect of using pytest: removal of some mocks that messed with it, no longer relying on a specific cwd, etc.

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed superseded WIP PR to run snapcraft unit tests with pytest (+xdist) sat idle and was closed unmerged; the migration later landed in #3167 (788475f62) after #3162 dropped testscenarios.
stealth/space-bunny-alpha — closed superseded WIP PR #2176 (pytest as test runner) was closed unmerged after stalling; the work was later redone and merged as #3167 (commit 788475f6), plus #3162 dropping testscenarios.
stealth/space-bunny-alpha — closed stale The pytest test-runner proposal was closed unmerged after more than a year of inactivity; a separate unit-test migration later landed in #3167 (788475f62).
qwen/qwen3.6-35b-a3b — — Abandoned. The pull request to switch to pytest as the test runner was closed without merging after over a year of inactivity, following discussions about parallel execution and test performance.
qwen3.6-35b-a3b-mtp-q6 — — Closed without merging. The work-in-progress pull request to adopt pytest was abandoned after discussions highlighted testscenarios causing long-running tests and CI timeouts.

Update history

No update history recorded yet.

Related work

  • Superseded By: canonical/snapcraft#3167 (confidence 95%)

    Merged commit 788475f62 'unit tests: move to pytest' performed the migration this WIP PR proposed.

  • Related To: canonical/snapcraft#3162 (confidence 88%)

    Merged commit 58c66b7a7 removed scenario usage from lifecycle order tests, the prerequisite cleanup named in this PR's comments.

Related issues

Issue Project State Summary Similarity
#3167 unit tests: move to pytest snapcraft merged Merged PR #3167 in commit 788475f62, migrating unit tests from testscenarios to pytest, adding fixtures and pytest-cov, and retaining unittest for integration tests.
72%