Reconsider python package dependency management
Metadata
Current evaluation
Legacy charm-plugin dependency redesign was superseded by dedicated Python and Poetry plugins, added in Charmcraft 3.3.0 by PR #1899 (8b748f3e).
Resolution: closed superseded
Reason: Dependency redesign was moved from the legacy charm plugin to dedicated Python and Poetry plugins. PR #1899, commit 8b748f3e, later implemented both plugins, which the 3.3.0 changelog records as new.
Evaluated by: mr-cal-bot
No scores available.
Issue body
### What needs to get done
Currently we run pip (up to) 4 times, and in a way that can be counterintuitive to the user. Package sets can override each other, and explicitly frozen package versions can be ignored.
Ideally, we'd have a way to specify this all to be calculated in a single `pip` command that:
- Only installs binary packages for those specified as binaries (and their dependencies?)
- Fails on package conflicts
- Doesn't overwrite pinned versions
### Why it needs to get done
The current method has evolved over years to become counterintuitive and sometimes problematic (see #1135). However, changes related to this could cause compatibility issues, so they should be done carefully and deliberately. As such, a rethink of this would need to come with a major version release if we decide to do it.
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed superseded | Legacy charm-plugin dependency redesign was superseded by dedicated Python and Poetry plugins, added in Charmcraft 3.3.0 by PR #1899 (8b748f3e). | |
| stealth/space-bunny-alpha | — | closed superseded | Dependency redesign was superseded by the Python and Poetry plugins, added in Charmcraft 3.3 through merged PR #1899 (commit 8b748f3e). | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed by PR #1233 (commit be5d6a5c), which introduced redesigned Python dependency management and replaced the legacy repeated-pip workflow. | |
| stealth/space-bunny-alpha | — | closed superseded | Closed as superseded: Python dependency management was redesigned through Charmcraft's Python and Poetry plugins, with migration guidance replacing changes to the legacy charm plugin. | |
| stealth/space-bunny-alpha | — | closed superseded | Dependency redesign moved to the Python and Poetry plugins; related issues #1135 and #1077 closed with migration guidance rather than changes to the legacy charm plugin. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed. The proposal to overhaul Python dependency management was deferred to a future major release. Related discussions and bugs were consolidated into issues #1192, #1058, and #1077. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed as a discussion and tracking issue. No fix implemented. Related dependency conflicts were linked to separate issues. The proposed pip management redesign was deferred pending a major version release. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without implementation. The proposal to overhaul pip dependency management was abandoned, with no concrete resolution provided. Related discussions and linked issues were referenced instead. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed without resolution, likely abandoned. Discussions covered pip conflicts, poetry integration, and resolver issues, but no dependency management changes were implemented. |
Update history
No update history recorded yet.
Related work
-
Likely Fixed By:
canonical/charmcraft#1899
(confidence 98%)
Merged commit 8b748f3e added the Python and Poetry plugins; both were released in Charmcraft 3.3.0.
-
Superseded By:
canonical/charmcraft#1873
(confidence 96%)
Dedicated Python and Poetry plugins replaced the proposed redesign of legacy charm-plugin dependency handling.
-
Related To:
canonical/charmcraft#1135
(confidence 95%)
Pinned-dependency override failure prompted the broader dependency-management rethink; PR #1157 only partially addressed it.
-
Related To:
canonical/charmcraft#1192
(confidence 92%)
Duplicate-requirement failure was discussed here and separately fixed in Charmcraft 2.4.1 by PR #1211.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1873 Create charmcraft-specific versions of the upstream `python` and `poetry` plugins | charmcraft | closed | Charmcraft-specific Python and Poetry plugins were added by merged PR #1899 (commit 8b748f3e), replacing the experimental work in #1840. | |
| #1843 Add charmcraft-specific versions of upstream python plugins | charmcraft | open | Request to add charmcraft-specific python/poetry plugins. Implemented by commit 8b748f3e (PR #1899) alongside #1873; the plugins are registered as 'python'/'poetry' in get_app_plugins() and have received later fixes (#1968, #2044, #2835). | |
| #1899 feat: add Python and Poetry plugins | charmcraft | merged | Merged into main via commit 8b748f3e, adding Charmcraft-specific Python and Poetry plugins with charm-oriented venv, pip, source-copy, and binary defaults. | |
| #1233 feat(dependencies): new Python dependency management | charmcraft | merged | Merged PR #1233 in commit be5d6a5c, adding single-step Python dependency management with fallback to Charmcraft 2.3.0 behavior and fixing #1140. | |
| #1077 Charms can import packages not installed in `venv` | charmcraft | closed | Charm-plugin behavior remains unchanged; Python and Poetry plugins add venv isolation, available in Charmcraft 3.3 via #1899 (8b748f3e). | |
| #553 parts,charm_builder: add charm-python-packages plugin property (CRAFT-552) | charmcraft | merged | Merged PR #553 added charm-python-packages, installing listed Python packages before requirements dependencies; implementation landed in commit 4b2d291b. | |
| #18 Poetry integration | charmcraft | closed | Poetry support was ultimately delivered as Charmcraft-specific python/poetry craft-parts plugins, added in PR #1899 (commit 8b748f3e, 2023), superseding the older build.py requirements.txt idea; issue closed once shipped. |