Python plugins fails to build a snap when some parts depends on unpublished modules
Metadata
Current evaluation
Fixed by PR #2771 (commit 9bac0fd64), which made the Python plugin try processing setup.py before querying PyPI, allowing parts to use locally built packages.
Resolution: closed resolved
Reason: The issue was resolved by PR #2771, whose commit is titled “python plugin: first try processing setup.py without PyPI.” The implementation was part of Snapcraft 3.10, as noted in the closing discussion.
Evaluated by: mr-cal-bot
No scores available.
Issue body
Hello,
I described the full case here: https://forum.snapcraft.io/t/building-a-snap-with-a-multi-python-project/12284
To sum up, let's suppose that you have two python projects, project1 and project2, lying in the same GIT repository. You are building a snap that consists in two parts, one for each project.
However, project2 depends on project1 in its current state, with new features added in project2 that are not yet release. Typically project2 will declare a dependency project1>=0.1.dev0.
Out of the snaps, this situation is OK for pip, since it will install project1 in the local site package, then will successfully install project2 since a compatible version of project1 is already available. However, snapcraft will fail in the same situation, complaining that it cannot find the required version of project1 in PyPI.
I checked the code of python plugin. In theory nothing prevents snapcraft to do the same thing than pip, since the sitecustomize module is properly set up to find packages from the other parts already compiled.
However, the root cause here is this line of code: https://github.com/snapcore/snapcraft/blob/6c3e3ab9acbf672760141f6456c3c47a92899fb3/snapcraft/plugins/python.py#L338
This line ask to invoke `pip download` for the set of requirements declared by a given part. And it will fail because it cannot find the required version for project1.
Funny thing is that in fact, the snap would be built perfectly without this, since the invocation of `pip install` after will successfully find the packages from the already installed snaps.
I made a showcase project (python projects + snapcraf.yml) that focuses on this error, if someone wants a clear target to fix this: https://github.com/adferrand/snapcraft-python-showcase
I am available for any question on that matter.
Regards,
Adrien Ferrand
Evaluation history
| Date | Model | Scores | Action | Summary |
|---|---|---|---|---|
| stealth/space-bunny-alpha | — | closed resolved | Fixed by PR #2771 (commit 9bac0fd64), which made the Python plugin try processing setup.py before querying PyPI, allowing parts to use locally built packages. | |
| stealth/space-bunny-alpha | — | closed superseded | Superseded by snapcore/snapcraft#2771, which proposed replacing the separate pip-download step so dependencies resolve against packages staged by earlier parts. | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed by merged PR #2771, commit 9bac0fd64: the Python plugin now checks local packages and requirements before falling back to PyPI. | |
| stealth/space-bunny-alpha | — | closed resolved | Fixed in snapcraft#2771 (commit 9bac0fd64): the Python plugin now tries processing setup.py without querying PyPI, allowing use of unpublished modules from other parts. | |
| qwen/qwen3.6-35b-a3b | — | — | Closed without resolution. The Python plugin fails to build snaps when parts depend on unpublished local modules because pip download fails to locate them on PyPI. No fix was implemented. | |
| qwen/qwen3.6-35b-a3b | — | — | The issue reported that the Python plugin fails to build snaps with unpublished local dependencies because pip download cannot find them on PyPI. The issue was closed with no activity, indicating it was abandoned. |
Update history
| Date | Change |
|---|---|
| created |
Related work
-
Likely Fixed By:
canonical/snapcraft#2771
(confidence 99%)
PR #2771 changed dependency processing to support packages provided by earlier parts instead of requiring them from PyPI.
Related issues
| Issue | Project | State | Summary | Similarity |
|---|---|---|---|---|
| #1794216 python plugin parts errors with "ERROR: You must give at least one requirement to install" when rebuilding | snapcraft (launchpad) | closed | Fixed Python plugin rebuilds involving cached wheels through snapcore/snapcraft#2498. | |
| #2161192 python plugin not working at all with snapcraft 9+ | snapcraft (launchpad) | open | Core26 Python plugin builds and module loading were fixed; packaging Python separately is now documented as required by the base. | |
| #6020 Most Python-based snaps can't build on core26 | snapcraft | closed | Fixed by merged PR #6089 (commit 6ee41173c) and released in Snapcraft 8.14.3, enabling Python plugin builds on core26. | |
| #1501997 python projects might fail to snap if the package python-dev is not installed | snapcraft (launchpad) | closed | Fixed by adding libpython2.7-dev as a Python plugin build package in commit d436aa787, originally proposed in e8fc4bd0f. | |
| #1860884 Python plugin: paths for from the snapcraft snap in sys.path break dependencies | snapcraft (launchpad) | closed | Snapcraft's Python plugin no longer leaks its site-packages into part builds. Fixed by snapcore/snapcraft#2901 (c13c79d76a) and released in Snapcraft 3.10. | |
| #1670852 python console_scripts not installed into /snap/<snap>/current/bin | snapcraft (launchpad) | closed | Fixed: regression in python plugin (from PR #1093 / commit e7824093f) where pip installed pbr/setup.cfg projects but skipped entry points. Fixed by PR #1202, commit dbdaf36a4, which runs setup.py instead of pip install. | |
| #1586546 snapcraft python2 plugin build fails because of .pth and PYTHONPATH test | snapcraft (launchpad) | closed | Fixed Python plugin setup.py handling in snapcore/snapcraft#718, merged as commit 810136457. | |
| #1631057 python plugin "No module named site" | snapcraft (launchpad) | closed | Fixed by snapcraft PR #858 (commit a2054b8a7), which made the Python plugin compatible with bzr-backed requirements. |