[charmcraft 1.3]: Cannot pack a charm with Jinja2 as dependency
Metadata
Current evaluation
Charmcraft 1.3 failed to pack charms with Jinja2 due to dependency issues. Closed with a setuptools workaround. Resolved via PR #553, which adds a charm-python-packages property to pre-install dependencies before requirements.
Suggested action: —
No scores available.
Issue body
Charmcraft 1.3 fails to pack our [slurmd charm](https://github.com/omnivector-solutions/slurm-charms). Reverting to 1.2 works again.
Here's part of the log:
```
[...]
2021-09-22 17:22:58,259 craft_parts.executor.step_handler DEBUG :: :: Collecting jinja2
2021-09-22 17:22:58,261 craft_parts.executor.step_handler DEBUG :: :: Using cached Jinja2-3.0.1.tar.gz (267 kB)
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: ERROR: Command errored out with exit status 1:
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: command: /root/parts/charm/build/staging-venv/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-nbidbuh_/jinja2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-nbidbuh_/jinja2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-install-nbidbuh_/jinja2/pip-egg-info
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: cwd: /tmp/pip-install-nbidbuh_/jinja2/
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: Complete output (35 lines):
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: Traceback (most recent call last):
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "<string>", line 1, in <module>
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/tmp/pip-install-nbidbuh_/jinja2/setup.py", line 4, in <module>
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: setup(
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/__init__.py", line 145, in setup
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: return distutils.core.setup(**attrs)
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/usr/lib/python3.8/distutils/core.py", line 121, in setup
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: dist.parse_config_files()
2021-09-22 17:22:58,388 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/dist.py", line 701, in parse_config_files
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: parse_configuration(self, self.command_options,
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 121, in parse_configuration
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: meta.parse()
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 426, in parse
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: section_parser_method(section_options)
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 399, in parse_section
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: self[name] = value
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 184, in __setitem__
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: value = parser(value)
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 515, in _parse_version
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: version = self._parse_attr(value, self.package_dir)
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/root/parts/charm/build/staging-venv/lib/python3.8/site-packages/setuptools/config.py", line 349, in _parse_attr
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: module = import_module(module_name)
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: return _bootstrap._gcd_import(name[level:], package, level)
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 991, in _find_and_load
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap_external>", line 848, in exec_module
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/tmp/pip-install-nbidbuh_/jinja2/src/jinja2/__init__.py", line 8, in <module>
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: from .environment import Environment as Environment
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: File "/tmp/pip-install-nbidbuh_/jinja2/src/jinja2/environment.py", line 15, in <module>
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: from markupsafe import Markup
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: ModuleNotFoundError: No module named 'markupsafe'
2021-09-22 17:22:58,389 craft_parts.executor.step_handler DEBUG :: :: ----------------------------------------
2021-09-22 17:22:58,401 craft_parts.executor.step_handler DEBUG :: :: ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
2021-09-22 17:22:58,427 craft_parts.executor.step_handler DEBUG :: Traceback (most recent call last):
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 345, in <module>
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: main()
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 341, in main
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: builder.build_charm()
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 93, in build_charm
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: self.handle_dependencies()
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 241, in handle_dependencies
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: _process_run(cmd)
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: File "/snap/charmcraft/x24/lib/charmcraft/charm_builder.py", line 292, in _process_run
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: raise CommandError(f"Subprocess command {cmd} execution failed with retcode {retcode}")
2021-09-22 17:22:58,428 craft_parts.executor.step_handler DEBUG :: charmcraft.cmdbase.CommandError: Subprocess command ['/root/parts/charm/build/staging-venv/bin/pip3', 'install', '--upgrade', '--no-binary', ':all:', '--requirement=requirements.txt'] execution failed with retcode 1
2021-09-22 17:22:58,446 charmcraft ERROR Parts processing error: Failed to run the build script for part 'charm'. (full execution logs in '/tmp/charmcraft.log')
2021-09-22 14:22:58,511 craft_providers.lxd.lxc DEBUG Executing in container: lxc --project charmcraft exec local:charmcraft-slurmd-11534686-0-0-amd64 -- env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin CHARMCRAFT_MANAGED_MODE=1 test -f /tmp/charmcraft.log
2021-09-22 14:22:58,598 craft_providers.lxd.lxc DEBUG Executing on host: lxc --project charmcraft file pull local:charmcraft-slurmd-11534686-0-0-amd64/tmp/charmcraft.log /home/h/snap/charmcraft/common/charmcraft-ib5f_flp
```
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| qwen3.6-35b-a3b-mtp-q6 | — | — | Charmcraft 1.3 failed to pack charms with Jinja2 due to dependency issues. Closed with a setuptools workaround. Resolved via PR #553, which adds a charm-python-packages property to pre-install dependencies before requirements. | |
| qwen/qwen3.6-35b-a3b | — | — | charmcraft 1.3 failed to pack charms with Jinja2 due to missing markupsafe. Resolved via PR #553, which adds a charm-python-packages property to pre-install dependencies like setuptools, fixing the build failure. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Closed. Charmcraft 1.3 failed to pack charms with Jinja2 due to missing markupsafe. Workaround: add setuptools via charm-python-packages. PR 553 proposes a new charm-python-packages property to install dependencies before requirements, addressing the root cause. | |
| qwen3.6-35b-a3b-mtp-q6 | — | — | Resolved by introducing a charm-python-packages property to the charm plugin, enabling pre-installation of dependencies like setuptools before requirements.txt processing. Implementation in PR #553. |
Update history
No update history recorded yet.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #632 Cant build charm with charmhelpers - missing markupsafe | charmcraft | closed | Charm build failed due to a setuptools version conflict between Jinja2 and Tempita. Resolved by adding charm-python-packages: [setuptools < 58] to charmcraft.yaml. Issue closed as resolved. | |
| #639 Can't build charm: Failed to build charm for bases index '0' | charmcraft | closed | Resolved as a jinja2 and setuptools version conflict. Provided a charmcraft.yaml workaround to install or pin setuptools. A later user confirmed charmcraft clean followed by pack also fixed it. Closed without code changes. | |
| #738 Crash with `charmcraft pack` | charmcraft | closed | Closed. The crash stemmed from outdated setuptools and pip failing to build jinja2. Resolved via a workaround specifying newer setuptools and pip in charmcraft.yaml. Maintainers attributed the error to upstream packaging and pip limitations. | |
| #922 Unable to pack charms on a GCP instance | charmcraft | closed | Closed as not a Charmcraft bug. The error stems from a pip and setuptools version mismatch. Maintainers provided a workaround: adding setuptools to charm-python-packages in charmcraft.yaml to force an update and resolve the build failure. | |
| #1781 charmcraft `3.1.1` "successfully" packs charm, but python packages are missing in the venv | charmcraft | closed | Missing Python packages in charmcraft 3.1.1 venvs resulted from a strict dependencies bug. The developer identified the missing pip check step, implemented a fix, and closed the report after updating validation logic. | |
| #1472 charmcraft 2.5.2 on PyPi doesn't install on jammy/py310 (at least) - error "ModuleNotFoundError: No module named 'tools'" | charmcraft | closed | Packaging error in charmcraft 2.5.2 caused install failures due to the missing tools module. The bug was fixed in Charmcraft 3.0. A backport to the 2.x branch was requested but not implemented. |