fix: make integration tests work on ARM Mac runners
Metadata
Current evaluation
ARM-Mac test fixes were abandoned: PR #2520 closed unmerged, and commit f8b267e reverted the faulty changes after review.
Resolution: closed stale
Reason: A reviewer reported that the changes worsened test failures, after which Copilot reverted them in f8b267e. The PR was closed without merging, and its proposed changes are absent from the current source.
Evaluated by: mr-cal-bot
No scores available.
Issue body
Integration tests were failing on ARM Macs (macos-15) due to incomplete test fixtures and architecture-specific test data.
## Changes Made
**tests/integration/services/sample_projects/complex-legacy/prime/manifest.yaml**
- Added missing 4th base (ubuntu 20.04) with 6 architectures to match actual code output
- The expected manifest was incomplete, causing test failures
**tests/integration/services/sample_projects/platforms-multi-base/project/charmcraft.yaml**
- Added `jammy-arm64` platform definition to support ARM64 builds
- Fixes `EmptyBuildPlanError` on ARM Macs by providing a valid build target
**tests/integration/services/test_package.py**
- Added `_normalize_architecture()` helper to adjust expected outputs for non-amd64 architectures
- Only replaces architecture when host architecture differs from amd64 and isn't already present
- Handles architecture differences without modifying fixture files
## Root Cause
The manifest files show `run-on` architectures from charmcraft.yaml, which are explicitly defined and don't vary by build host. The test failures were caused by:
1. Incomplete expected manifest (complex-legacy missing a base entry)
2. Missing ARM64 platform definition in platforms-multi-base test project
## Testing
- ✅ Code review passed
- ✅ Security scan passed (no vulnerabilities found)
- ⏳ Awaiting CI validation on macos-15 runners
---
- [ ] I've followed the [contribution guidelines](https://github.com/canonical/charmcraft/blob/main/CONTRIBUTING.md).
- [ ] I've signed the [CLA](http://www.ubuntu.com/legal/contributors/).
- [ ] I've successfully run `make lint && make test`.
- [ ] I've added or updated any relevant documentation.
- [ ] I've updated the relevant release notes.
<!-- START COPILOT CODING AGENT TIPS -->
---
✨ Let Copilot coding agent [set things up for you](https://github.com/canonical/charmcraft/issues/new?title=✨+Set+up+Copilot+instructions&body=Configure%20instructions%20for%20this%20repository%20as%20documented%20in%20%5BBest%20practices%20for%20Copilot%20coding%20agent%20in%20your%20repository%5D%28https://gh.io/copilot-coding-agent-tips%29%2E%0A%0A%3COnboard%20this%20repo%3E&assignees=copilot) — coding agent works faster and does higher quality work when set up for your repo.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed stale | ARM-Mac test fixes were abandoned: PR #2520 closed unmerged, and commit f8b267e reverted the faulty changes after review. | |
| stealth/space-bunny-alpha | — | closed stale | PR #2520 was closed unmerged. Its attempted ARM-Mac test fixes were reverted in f8b267e after a reviewer reported they worsened the failures. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without merge. Changes to fix ARM Mac integration tests were reverted after reviewer feedback indicated they worsened the issue. The contributor identified missing expected manifest entries as the root cause but did not resubmit a working fix. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without merging after a reviewer noted regressions. The author reverted template changes and determined updating expected manifest fixtures, rather than runtime logic, fixes ARM Mac test failures. |
Update history
No update history recorded yet.
Related work
-
Superseded By:
canonical/charmcraft#2606
(confidence 85%)
A later CI change disabled macOS tests, replacing the need for these abandoned ARM-specific fixture edits.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #596 ci: enable tests on arm mac | craft-providers | closed | PR #596 proposed enabling CI tests on ARM Mac but was closed unmerged after 785 days without reviews, comments, checks, or a resolving commit. | |
| #2491 ci: switch macos tests from 13 to 15 | charmcraft | closed | Closed unmerged; ARM-Mac fixes moved to #2520, which was also closed unmerged after f8b267e reverted the faulty changes. | |
| #971 tests: fix integration tests in armhf | snapcraft | merged | Merged PR #971 as commit a725e69da, fixing armhf integration-test setup with architecture-specific test configuration. | |
| #917 Fix unittests in armhf | snapcraft | closed | armhf unit-test fixes PR closed unmerged by come-maiz in favor of #945, which split integration and snaps tests onto separate branches; #945 was later merged as commit 073332b9b. | |
| #77 ci: remove broken macos integration tests | craft-application | merged | Merged PR #77 (commit 37f90c80) removed broken macOS integration tests from CI; re-enabling them was deferred until the image is fixed. | |
| #145 fix: fix tests on arm64 | craft-archives | merged | Merged as commit 8459563, fixing arm64 test expectations after prerequisite #143 and resolving #144. |