Charmcraft 3.x not including symlinked directories in resulting charm
Metadata
Current evaluation
Charmcraft 3.x dropped symlinked directories (e.g. hooks/charmhelpers) from charm ZIPs due to Path.rglob() not following dir symlinks. Fixed by PR #1745, commit ac08e98e, which restored 2.x os.walk(followlinks=True) behaviour; shipped in Charmcraft 3.1.0.
Resolution: closed resolved
Reason: Merged PR #1745 (commit ac08e98e, 'fix(zip): include linked directories in created zips') replaced rglob with os.walk(followlinks=True) when collecting prime-dir files, restoring symlinked directory inclusion, and the fix is listed in the 3.1.0 changelog. Maintainer @lengau confirmed the root cause as the rglob vs os.walk symlink difference.
Evaluated by: mr-cal-bot
No scores available.
Issue body
### Bug Description
Building classical charms with version 3 is not adding the submodules like charmhelpers in the hooks folder.
So running a charm build with version 3 is breaking saying that it's not possible to find the module charmhelpers
### To Reproduce
install charmcraft latest/edge
build a classic charm like [charm-nrpe](https://github.com/canonical/charm-nrpe) and deploy.
### Environment
Ubuntu 22.04 LTS
### charmcraft.yaml
```shell
type: charm
parts:
charm:
plugin: dump
source: .
prime:
- actions/*
- files/*
- hooks/*
- hooks/charmhelpers/*
- templates/*
- actions.yaml
- config.yaml
- copyright
- LICENSE
- metadata.yaml
- README.md
bases:
- build-on:
- name: ubuntu
channel: "20.04"
architectures: ["amd64"]
run-on:
- name: ubuntu
channel: "22.04"
architectures:
- amd64
- name: ubuntu
channel: "20.04"
architectures:
- amd64
- name: ubuntu
channel: "18.04"
architectures:
- amd64
```
### Relevant log output
```shell
unit-nrpe-0: 12:44:22 INFO juju.worker.meterstatus skipped "meter-status-changed" hook (missing)
unit-nrpe-0: 12:44:22 WARNING unit.nrpe/0.install Traceback (most recent call last):
unit-nrpe-0: 12:44:22 WARNING unit.nrpe/0.install File "/var/lib/juju/agents/unit-nrpe-0/charm/hooks/install", line 4, in <module>
unit-nrpe-0: 12:44:22 WARNING unit.nrpe/0.install import services
unit-nrpe-0: 12:44:22 WARNING unit.nrpe/0.install File "/var/lib/juju/agents/unit-nrpe-0/charm/hooks/services.py", line 11, in <module>
unit-nrpe-0: 12:44:22 WARNING unit.nrpe/0.install from charmhelpers.core import hookenv
unit-nrpe-0: 12:44:22 WARNING unit.nrpe/0.install ModuleNotFoundError: No module named 'charmhelpers'
unit-nrpe-0: 12:44:22 ERROR juju.worker.uniter.operation hook "install" (via explicit, bespoke hook script) failed: exit status 1
unit-nrpe-0: 12:44:22 INFO juju.worker.uniter awaiting error resolution for "install" hook
unit-nrpe-0: 12:45:49 INFO juju.worker.uniter awaiting error resolution for "install" hook
unit-nrpe-0: 12:48:43 INFO juju.worker.uniter awaiting error resolution for "install" hook
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Charmcraft 3.x dropped symlinked directories (e.g. hooks/charmhelpers) from charm ZIPs due to Path.rglob() not following dir symlinks. Fixed by PR #1745, commit ac08e98e, which restored 2.x os.walk(followlinks=True) behaviour; shipped in Charmcraft 3.1.0. | |
| stealth/space-bunny-alpha | — | closed resolved | Charmcraft 3.x omitted symlinked directories from classic charm zips. Commit ac08e98e, merged in #1745, fixed linked-directory inclusion. | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed by merged PR #1745 (ac08e98e), restoring build_zip directory-symlink traversal so linked directories such as hooks/charmhelpers are included; the PR fixes #1719. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without a fix. The root cause was identified as a behavioral difference between Path.rglob() and os.walk() regarding directory symlinks. The report was tracked internally as CRAFT-3135. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed and tracked internally as CRAFT-3135. Root cause identified as a Path.rglob() versus os.walk() symlink handling difference. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed after identifying the root cause as a behavioral difference between Path.rglob() and os.walk() regarding directory symlinks. The issue was resolved without a code fix, documenting the symlink traversal limitation. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed after root cause analysis revealed Path.rglob() and os.walk() handle directory symlinks differently. Tracked via Jira CRAFT-3135 for resolution. |
Update history
No update history recorded yet.
Related work
-
Likely Fixed By:
canonical/charmcraft#1745
(confidence 97%)
Merged PR 'fix(zip): include linked directories in created zips' — commit ac08e98e; directly resolves the reported rglob/os.walk symlink regression.
-
Related To:
canonical/charmcraft#1968
(confidence 25%)
Later charmcraft fix for duplicate files in built charms touching the same zip/prime collection code path, not directly this issue.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1745 fix(zip): include linked directories in created zips | charmcraft | merged | Merged #1745 (commit ac08e98e) restored Charmcraft 2.x symlink-following behavior, including linked directories in generated charm and bundle ZIPs; documented in the 3.1.0 changelog. | |
| #260 Can't include symlinked file into built charm | charmcraft | closed | Resolved by Charmcraft’s craft-parts build flow in PR #458 (implementation 437dab55, merge 9dc20872), enabling build-time handling of linked project files. |