← Back to issue list

[charmcraft 1.3]: Cannot pack a charm with Jinja2 as dependency

View original Github issue

Metadata

Project
charmcraft
Number
#551
Type
issue
State
closed
Author
heitorPB
Labels
—
Created
Updated
Closed

Current evaluation

Jinja2 pack failure in charmcraft 1.3 was traced to old setuptools in the build venv. Resolved by merged PR #553 (ccec8b7e, commit 4b2d291b) adding a charm-python-packages charm plugin property, letting charms preinstall setuptools.

Resolution: closed resolved

Reason: PR #553 "parts,charm_builder: add charm-python-packages plugin property (CRAFT-552)" was merged as ccec8b7e, adding charm-python-packages; maintainer cmatsuoka closed the issue noting the Jinja2 pack issue is fixed by using `charm-python-packages: [setuptools]`.

Evaluated by: mr-cal-bot

No scores available.

Issue body

Charmcraft 1.3 fails to pack our [slurmd charm](https://github.com/omnivector-solutions/slurm-charms). Reverting to 1.2 works again. Here's part of the log: ``` [...] 2021-09-22 17:22:58,259 craft_parts.executor.step_handler DEBUG :: :: Collecting jinja2 2021-09-22 17:22:58,261 craft_parts.executor.step_handler DEBUG :: :: Using cached Jinja2-3.0.1.tar.gz (267 kB) 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: ERROR: Command errored out with exit status 1: 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: command: /root/parts/charm/build/staging-venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nbidbuh_/jinja2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nbidbuh_/jinja2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-nbidbuh_/jinja2/pip-egg-info 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: cwd: /tmp/pip-install-nbidbuh_/jinja2/ 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: Complete output (35 lines): 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: Traceback (most recent call last): 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "<string>", line 1, in <module> 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/tmp/pip-install-nbidbuh_/jinja2/setup.py", line 4, in <module> 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: setup( 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: return distutils.core.setup(**attrs) 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/usr/lib/python3.8/distutils/core.py", line 121, in setup 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: dist.parse_config_files() 2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/dist.py", line 701, in parse_config_files 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: parse_configuration(self, self.command_options, 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 121, in parse_configuration 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: meta.parse() 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 426, in parse 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: section_parser_method(section_options) 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 399, in parse_section 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: self[name] = value 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 184, in __setitem__ 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: value = parser(value) 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 515, in _parse_version 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: version = self._parse_attr(value, self.package_dir) 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 349, in _parse_attr 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: module = import_module(module_name) 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: return _bootstrap._gcd_import(name[level:], package, level) 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 991, in _find_and_load 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap_external>", line 848, in exec_module 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/tmp/pip-install-nbidbuh_/jinja2/src/jinja2/__init__.py", line 8, in <module> 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: from .environment import Environment as Environment 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/tmp/pip-install-nbidbuh_/jinja2/src/jinja2/environment.py", line 15, in <module> 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: from markupsafe import Markup 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: ModuleNotFoundError: No module named 'markupsafe' 2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: ---------------------------------------- 2021-09-22 17:22:58,401 craft_parts.executor.step_handler DEBUG :: :: ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output. 2021-09-22 17:22:58,427 craft_parts.executor.step_handler DEBUG :: Traceback (most recent call last): 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 345, in <module> 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: main() 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 341, in main 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: builder.build_charm() 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 93, in build_charm 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: self.handle_dependencies() 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 241, in handle_dependencies 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: _process_run(cmd) 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 292, in _process_run 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: raise CommandError(f"Subprocess command {cmd} execution failed with retcode {retcode}") 2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: charmcraft.cmdbase.CommandError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip3', 'install', '--upgrade', '--no-binary', ':all:', '--requirement=requirements.txt'] execution failed with retcode 1 2021-09-22 17:22:58,446 charmcraft ERROR Parts processing error: Failed to run the build script for part 'charm'. (full execution logs in '/tmp/charmcraft.log') 2021-09-22 14:22:58,511 craft_providers.lxd.lxc DEBUG Executing in container: lxc --project charmcraft exec local:charmcraft-slurmd-11534686-0-0-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CHARMCRAFT_MANAGED_MODE=1 test -f /tmp/charmcraft.log 2021-09-22 14:22:58,598 craft_providers.lxd.lxc DEBUG Executing on host: lxc --project charmcraft file pull local:charmcraft-slurmd-11534686-0-0-amd64/tmp/charmcraft.log /home/h/snap/charmcraft/common/charmcraft-ib5f_flp ```

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed resolved Jinja2 pack failure in charmcraft 1.3 was traced to old setuptools in the build venv. Resolved by merged PR #553 (ccec8b7e, commit 4b2d291b) adding a charm-python-packages charm plugin property, letting charms preinstall setuptools.
stealth/space-bunny-alpha — closed resolved Jinja2 build failure in charmcraft 1.3 traced to outdated setuptools; fixed by merged PR #553 (commit4b2d291b, merge ccec8b7e) adding charm-python-packages, letting charms preinstall setuptools/pip.
stealth/space-bunny-alpha — closed resolved Merged PR #553 (ccec8b7e) added charm-python-packages, enabling setuptools to be preinstalled and restoring Jinja2 charm packing with charmcraft 1.3.
qwen3.6-35b-a3b-mtp-q6 — — Charmcraft 1.3 failed to pack charms with Jinja2 due to dependency issues. Closed with a setuptools workaround. Resolved via PR #553, which adds a charm-python-packages property to pre-install dependencies before requirements.
qwen/qwen3.6-35b-a3b — — charmcraft 1.3 failed to pack charms with Jinja2 due to missing markupsafe. Resolved via PR #553, which adds a charm-python-packages property to pre-install dependencies like setuptools, fixing the build failure.
qwen3.6-35b-a3b-mtp-q6 — — Closed. Charmcraft 1.3 failed to pack charms with Jinja2 due to missing markupsafe. Workaround: add setuptools via charm-python-packages. PR 553 proposes a new charm-python-packages property to install dependencies before requirements, addressing the root cause.

Update history

No update history recorded yet.

Related work

  • Likely Fixed By: canonical/charmcraft#553 (confidence 95%)

    Merged PR adding charm-python-packages property so setuptools can be installed before requirements.

  • Related To: jinja#1496 (confidence 60%)

    Upstream Jinja issue about older Jinja versions failing to install, referenced by maintainer facundobatista.

Related issues

Issue Project State Summary Similarity
#639 Can't build charm: Failed to build charm for bases index '0' charmcraft closed Closed as a dependency/configuration issue: adding setuptools fixed Jinja2 packing, while Tempita required setuptools <58. A later build recovered after charmcraft clean.
78%
#738 Crash with `charmcraft pack` charmcraft closed Closed as an upstream Jinja2/setuptools packaging interaction; removing Jinja2 or preinstalling newer pip and setuptools through charm-python-packages restored packing.
77%
#566 Error when building with charmcraft from edge charmcraft closed Closed by facundobatista as a duplicate of #551 (Jinja2/setuptools packing failure in 1.3 edge). Resolved via merged PR #553 (ccec8b7e, commit 4b2d291b) adding charm-python-packages; workaround is adding setuptools/pip.
77%
#922 Unable to pack charms on a GCP instance charmcraft closed Closed as not a Charmcraft bug: Focal's old setuptools broke pip's wheel installation. Preinstalling setuptools via charm-python-packages works around it; #923 tracked the same regression.
72%
#632 Cant build charm with charmhelpers - missing markupsafe charmcraft closed Charmhelpers-based charm build failed with missing markupsafe via Jinja2/setuptools 58+ vs Tempita 2to3 conflict. cmatsuoka diagnosed it and the reporter confirmed the workaround works; maintainer closed the issue as solved without a charmcraft code change.
72%
#1781 charmcraft `3.1.1` "successfully" packs charm, but python packages are missing in the venv charmcraft closed Strict-dependency packing now runs pip check and rejects inconsistent venvs; fixed in PR #1784 (fd21df24) and backported to 3.1 via PR #1794 (1ac1aa59).
72%
#553 parts,charm_builder: add charm-python-packages plugin property (CRAFT-552) charmcraft merged Merged PR #553 added charm-python-packages, installing listed Python packages before requirements dependencies; implementation landed in commit 4b2d291b.
71%