← Back to issue list

ci: separate flaky tests into their own job

View original Github issue

Metadata

Project
craft-parts
Number
#1583
Type
pull request
State
merged
Author
lengau
Labels
—
Created
Updated
Closed

Current evaluation

Merged PR #1583 in commit ec6458af, separating @pytest.mark.flaky tests into a dedicated test-flaky CI job for quick, independent reruns.

Resolution: closed resolved

Reason: Approved PR #1583 was merged as commit ec6458af. The workflow now excludes flaky tests from test-common and runs them independently through the test-flaky job.

Evaluated by: mr-cal-bot

No scores available.

Issue body

Move tests marked with `@pytest.mark.flaky` into a dedicated `test-flaky` CI job so they can be rerun independently and quickly when they fail due to transient issues (e.g. Ubuntu archive syncs). This avoids needing to re-run the full suite of slow tests if flaky tests fail. Artifact upload failure is fixed in https://github.com/canonical/starflow/pull/139

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed resolved Merged PR #1583 in commit ec6458af, separating @pytest.mark.flaky tests into a dedicated test-flaky CI job for quick, independent reruns.
stealth/space-bunny-alpha — closed resolved Merged CI changes moving @pytest.mark.flaky tests into a dedicated test-flaky job for fast, independent reruns. Coverage artifact upload collisions were fixed separately in canonical/starflow#139.
qwen/qwen3.6-35b-a3b — — Merged PR isolates flaky tests into a dedicated test-flaky CI job, enabling independent reruns and avoiding full suite re-executions. Approved by two reviewers and merged.

Update history

No update history recorded yet.

Related work

  • Related To: canonical/starflow#139 (confidence 100%)

    PR #1583 cites this separate fix for CI artifact-upload failures.

Related issues

Issue Project State Summary Similarity
#1088 ci: separate flaky tests into their own run craft-application merged Merged PR #1088 in commit 6d755f00, splitting flaky slow tests into dedicated test-flaky CI runs for quicker reruns during network outages.
85%