← Back to issue list

Please allow for installing Python dependencies from requirements files using wheels

View original Github issue

Metadata

Project
charmcraft
Number
#712
Type
issue
State
closed
Author
Vultaire
Labels
—
Created
Updated
Closed

Current evaluation

Maintainers declined wheel-based requirements installs to keep builds controlled and source-tracked, directing binary corner cases to charm-binary-python-packages.

Resolution: closed not a bug

Reason: In the closing response on 2022-06-03, facundobatista said there were no plans to allow binary requirements installs by default because source builds keep deployed code trackable, and recommended charm-binary-python-packages for binary corner cases.

Evaluated by: mr-cal-bot

No scores available.

Issue body

At present, packages from requirements files are installed via pip using the ```--no-binary :all:``` flag. This can result in exceptionally long build times which, if the charm author is OK with using pre-built wheels, are unnecessarily long. I'm working on a charm, and its requirements.txt looks like this: ``` charmhelpers ops click fabric python-openstackclient ``` Building the venv for the above requirements takes under 30 seconds on my machine. (Tested via both Focal and Bionic LXDs, using ```time { python3 -m venv venv; . venv/bin/activate; pip install --upgrade setuptools pip; pip install --upgrade -r requirements.txt; }```.) I would expect charmcraft.yaml to look something like this: ``` type: "charm" bases: - name: "ubuntu" channel: "20.04" architectures: - amd64 - name: "ubuntu" channel: "18.04" architectures: - amd64 parts: charm: charm-binary-python-packages: # Updated versions needed for working nicely with # jinja2/markupsafe - setuptools - pip ``` However, the packages in the requirements.txt depend on a wide range of other packages, and with the ```--no-binary :all:``` flag, it takes a very long time - longer than I want to clock for this bug. As an attempt at working around this issue, I've customized my charmcraft.yaml to look like this instead: ``` parts: charm: build-packages: # Needed by cryptography - libffi-dev - rustc - cargo - libssl-dev charm-binary-python-packages: # Updated versions needed for working nicely with # jinja2/markupsafe - setuptools - pip # Needed as prerequisites for other required packages. # Explicitly included here to avoid doing source-based builds # which take time and require installing extra dev packages. - setuptools-rust #- cryptography # note: seems to require a source build on Bionic, # which in turn requires setuptools-rust to be installed # before attempting to install this. - bcrypt - pynacl # note: no binary on bionic, but this does help focal # Include any of the requirements.txt packages which have wheels #- charmhelpers # currently tarball-only - ops - click #- fabric # currently tarball-only #- python-openstackclient # pulls cryptography dependency before # setuptools-rust can be installed ``` While this does install a few more bits via binaries when possible, my Bionic build time from venv creation through final dependency install (and excluding the other parts of the charm building process) takes roughly 13.5 minutes. Again, doing this via a normal venv only takes me roughly 30 seconds since most of the packages can be installed via wheels. Long story short: I don't see a good workaround at present to get reasonable build times for my particular required packages when using Charmcraft 1.4.0. Can we add an option so we can omit the ```--no-binary :all:``` when installing from requirements.txt files, or something similar in nature? Test environment: Host: Ubuntu 20.04 Charmcraft: latest/stable (1.4.0) Invoked via: ```charmcraft -v pack``` Building from bases: 20.04 and 18.04

Evaluation history

Date Model Scores Action Summary
stealth/space-bunny-alpha — closed not a bug Maintainers declined wheel-based requirements installs to keep builds controlled and source-tracked, directing binary corner cases to charm-binary-python-packages.
stealth/space-bunny-alpha — closed not a bug Closed as intended behavior: requirements remained source-built for build control and reproducibility, with selective binaries available via charm-binary-python-packages.
stealth/space-bunny-alpha — closed not a bug Wheel use for all requirements was declined; maintainers retained source builds and directed corner cases to charm-binary-python-packages.
stealth/space-bunny-alpha — closed not a bug Wheel support for requirements was declined to preserve source-based, reproducible builds. Maintainers directed users to specify corner-case binaries with charm-binary-python-packages.
qwen3.6-35b-a3b-mtp-q6 — — Rejected. Maintainers declined to enable wheel installation by default, prioritizing reproducible source builds. Users can manually specify individual packages for binary installation via the charm-binary-python-packages config key.
qwen/qwen3.6-35b-a3b — — Rejected. Maintainers declined to enable wheel installation by default, maintaining the --no-binary :all: policy to ensure controlled, traceable source builds. Users are directed to use charm-binary-python-packages for specific dependencies instead.
qwen3.6-35b-a3b-mtp-q6 — — The request was declined. Maintainers prioritize source builds for reproducibility. Users can enable wheel installation for specific packages by listing them in the charm-binary-python-packages configuration key.

Update history

No update history recorded yet.

Related issues

Issue Project State Summary Similarity
#342 When installing requirements, pip should avoid using wheels charmcraft closed Fixed in Charmcraft 1.3 by merged PR #541 (commit 0d14d8a2), which disabled binary packages so dependencies are built from source during charm packing.
75%