macOS CI is failing in PRs
Metadata
Current evaluation
macOS PR CI fails without logs; qa.yaml multipass/smoketest jobs commented out as a temporary workaround (#982), but root cause (runner timeout/disconnect) unfixed and weekly macOS jobs still fail per #467. Reopened by bepri.
Suggested action: keep open
Reason: Root cause remains unresolved: the only remediation is commit a3af10b "ci: temporarily disable macos smoke tests (#982)", which merely comments out the test-multipass and macos-smoketest jobs in .github/workflows/qa.yaml, and tests-weekly.yaml still runs integration-macos/integration-slow-macos on macos-14-large with the disconnect symptom described in #467. bepri reopened on 2026-07-01 noting it was "mistakenly closed by GitHub", so the earlier closure is not evidence of a fix. Diagnosis requires access to GitHub-hosted runners (maintainer's stated next step: "ssh into a runner and try clearing up some disk space") and there are no logs, which keeps complexity and actionability moderate despite the triaged status.
Evaluated by: mr-cal-bot
Issue body
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha |
Impact:
50
Quick Win:
7
Complexity:
75
Confidence:
82
Actionability:
55
|
keep open | macOS PR CI fails without logs; qa.yaml multipass/smoketest jobs commented out as a temporary workaround (#982), but root cause (runner timeout/disconnect) unfixed and weekly macOS jobs still fail per #467. Reopened by bepri. | |
| stealth/space-bunny-alpha |
Impact:
45
Quick Win:
8
Complexity:
65
Confidence:
90
Actionability:
50
|
keep open | Triaged macOS PR CI failures remain open after an accidental closure; a maintainer observed runner timeouts, and only a temporary smoke-test workaround has landed. | |
| qwen3.6-35b-a3b-mtp-q6 |
Impact:
60
Quick Win:
36
Staleness:
65
Complexity:
40
Confidence:
70
Support Request:
10
|
keep open | macOS CI failing in PRs due to runner timeouts or resource exhaustion; PR macOS CI currently disabled via commented-out jobs in qa.yaml | |
| qwen/qwen3.6-35b-a3b |
Staleness:
15
Complexity:
40
Confidence:
90
Support Request:
5
|
keep open | macOS CI tests fail without logs, likely due to runner timeouts or resource exhaustion. Recently reopened and actively debugged by maintainers. | |
| qwen3.6-35b-a3b-mtp-q6 |
Staleness:
25
Complexity:
40
Confidence:
85
Support Request:
10
|
keep open | macOS CI tests are timing out and failing without logs, likely due to runner resource limits. Recently reopened after being mistakenly closed and is under investigation. |
Update history
No update history recorded yet.
Related work
-
Related To:
craft-providers#467
(confidence 88%)
Weekly-test twin of the same macOS runner failure (~60 min disconnect, lost logs); #467 states PR CI was stabilized by #458–#460 while the underlying runner problem remains.
-
Related To:
craft-providers#982
(confidence 80%)
PR 'ci: temporarily disable macos smoke tests' (commit a3af10b) is the workaround that silenced PR CI by commenting out the macOS jobs, not a fix.
-
Related To:
craft-providers#458
(confidence 60%)
Commit f2d603e 'tests: use smoke integration tests for macOS' was the earlier PR-CI stabilization attempt later superseded by the disable commit.
-
Related To:
craft-providers#319
(confidence 55%)
Merged PR 'ci: reduce use of macos in CI' shows a long-running pattern of mitigating macOS CI by reducing its scope rather than fixing the runners.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #467 macOS CI is failing weekly tests | craft-providers | open | Maintainer-tracked CI issue: macOS runners drop connection after ~1h losing all logs, so flaky macOS integration tests were pushed out of PR CI (#452) into the weekly workflow. Still open, no logs or reproducer captured. | |
| #982 ci: temporarily disable macos smoke tests | craft-providers | merged | Merged PR #982 temporarily disabled macOS smoke-test CI after external runner failures; the underlying problem remains tracked in #452. |