tests: move the catkin integration tests to a separate suite
Metadata
Current evaluation
Merged in snapcraft#1760 as commit c97c45131, moving Catkin integration tests to a separate suite to reduce timeouts in the main plugins suite.
Resolution: closed resolved
Reason: PR #1760 was approved and merged as commit c97c45131. The change separated the long-running Catkin integration tests from the main plugins suite.
Evaluated by: mr-cal-bot
No scores available.
Issue body
- [x] Have you followed the [guidelines for contributing](https://github.com/snapcore/snapcraft/blob/master/CONTRIBUTING.md)?
- [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [x] If this is a bugfix. Have you checked that there is a bug report open for the issue you are trying to fix on [bug reports](https://bugs.launchpad.net/snapcraft)?
- [x] If this is a new feature. Have you discussed the design on the [forum](https://forum.snapcraft.io)?
- [x] Have you successfully run `./runtests.sh static`?
- [x] Have you successfully run `./runtests.sh unit`?
-----
Applying the results of the discussion with @kyrofa earlier this week. This is not a nice solution, but none of the alternatives were nice either. We have some grand ideas to make the suite dynamic, but that's very complex.
I think that we will start hitting the timeouts often in the remaining plugins suite, so we probably will have to do the same for the python or ruby tests.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Merged in snapcraft#1760 as commit c97c45131, moving Catkin integration tests to a separate suite to reduce timeouts in the main plugins suite. | |
| stealth/space-bunny-alpha | — | closed resolved | Merged as commit c97c45131: catkin integration tests moved to a separate suite, reducing main-suite timeouts. | |
| qwen/qwen3.6-35b-a3b | — | — | Merged after relocating catkin integration tests to a dedicated suite to prevent main suite timeouts. Approved by one reviewer, passed CI checks, and modified eight files. Similar refactoring planned for Python and Ruby tests. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged changes moving catkin integration tests to a separate suite to prevent main plugin suite timeouts, with plans to apply similar restructuring to python and ruby tests. |
Update history
No update history recorded yet.
Related work
-
Related To:
snapcraft#1791
(confidence 95%)
Follow-up PR #1791 applied the same suite-splitting approach to Python plugin integration tests.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #2206 tests: add spread suite for catkin plugin | snapcraft | merged | PR #2206 merged as commit 317d4a03a, converting Catkin plugin and snapd integration tests into a Spread suite and resolving LP bug #1783828. | |
| #2989 tests: only run catkin based snap on 16.04 | snapcraft | merged | Merged in #2989 (commit 7e1b62231), restricting the catkin-based snap test to Ubuntu 16.04 to prevent timeouts on 18.04. | |
| #1716 tests: split the integration autopkgtests | snapcraft | merged | Merged in snapcraft#1716 via 57af155a7, splitting integration autopkgtests into separate runs to prevent timeout failures despite added setup overhead. | |
| #1783828 Move Catkin plugin tests to spread | snapcraft (launchpad) | closed | Catkin plugin integration tests were migrated from Python to a Spread suite by merged PR #2206, commit 317d4a03a. | |
| #175 Catkin plugin: Refactor build. | snapcraft | merged | Merged in snapcraft#175 via commit 9045874fd, refactoring Catkin environment, dependency handling, build logic, source-space support, and tests. | |
| #1783829 Move Catkin Tools plugin tests to spread | snapcraft (launchpad) | closed | Catkin Tools integration coverage was migrated from Python to Spread by merged canonical/snapcraft#2206 (317d4a03a); LP #1783829 was later closed as stale housekeeping. | |
| #1948 tests: move test files out of the snapcraft dir | snapcraft | merged | Merged as commit 55ad11ccc: a pure test-suite reorganization (788 files, +615/-589) moving test files out of the snapcraft/ package dir into top-level tests/; approved by a maintainer and now reflected in the current tests/unit layout. | |
| #1791 tests: python plugin integration tests moved to a separate suite | snapcraft | merged | Merged PR #1791 (commit 91b6639f2) by come-maiz: Python plugin integration tests were moved out of the oversized plugins suite into their own suite, following the same split already done for catkin in #1760. |