unit tests: make the check for output less strict
Metadata
Current evaluation
Merged following approval and passing CI. Unit test output assertions were relaxed to verify message content and format rather than exact strings, accommodating rogue log messages generated during log level setup.
Suggested action: —
No scores available.
Issue body
We need to be less strict given that rogue messages show up when
setting up the log level. We now check for the messages and the
format we want instead of exact output.
Signed-off-by: Sergio Schvezov <sergio.schvezov@canonical.com>
- [x] Have you followed the [guidelines for contributing](https://github.com/snapcore/snapcraft/blob/master/CONTRIBUTING.md)?
- [x] Have you signed the [CLA](http://www.ubuntu.com/legal/contributors/)?
- [x] If this is a bugfix. Have you checked that there is a bug report open for the issue you are trying to fix on [bug reports](https://bugs.launchpad.net/snapcraft)?
- [x] If this is a new feature. Have you discussed the design on the [forum](https://forum.snapcraft.io)?
- [x] Have you successfully run `./runtests.sh static`?
- [x] Have you successfully run `./runtests.sh unit`?
-----
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen/qwen3.6-35b-a3b | — | — | Merged following approval and passing CI. Unit test output assertions were relaxed to verify message content and format rather than exact strings, accommodating rogue log messages generated during log level setup. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged. Relaxes unit test output assertions to accommodate unexpected log messages during setup, checking for message presence and format instead of exact string matches. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Merged. Relaxed unit test output validation to verify message format instead of exact strings, accommodating unexpected log messages generated during log level initialization. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #2637 unit tests: make lifecycle tests more robust | snapcraft | merged | Merged updates to improve lifecycle unit test robustness. Replaced strict message assertions with flexible checks to prevent fake webserver output from causing false failures. Approved and passed CI. | |
| #583 Rewrite of all tests that assert outputs to improve messages testing using new machinery. | charmcraft | merged | Merged a maintainer-led rewrite of output assertion tests to improve message testing via new machinery. Approved by one reviewer with no unresolved comments. Modified 17 files, adding 639 and removing 791 lines. | |
| #757 fix(testing): improve testing messages | craft-application | merged | Merged after two approvals and passing all CI checks. The update improves testing success reporting and provides clearer error messages on failure. |