[focal/core20][python3.7+] staging conflicts when multiple python parts have the same python dependencies

Metadata

Project: snapcraft (launchpad)
Number: #1882535
Type: issue
State: open
Author: ~dmitriis
Labels: —
Created: 2020-06-08 01:07 PM UTC
Updated: 2023-11-11 03:03 AM UTC
Closed: —

Current evaluation

Overlapping Python dependencies generate nondeterministic bytecode, RECORD, and venv artifacts that block staging; confirmed to affect multiple users, with workarounds but no complete fix.

Suggested action: keep open

Reason: The bug was explicitly marked Confirmed after maintainers noted that it affects multiple users. Although a maintainer said it would be fixed in 4.1, the reporter showed that SOURCE_DATE_EPOCH alone was insufficient because pip embeds temporary extraction paths in code objects, and the current codebase has no SOURCE_DATE_EPOCH or bytecode-deduplication handling. A robust design must address bytecode, RECORD files, and venv metadata while preserving deterministic builds.

Evaluated by: mr-cal-bot

Impact: 65 Quick Win: 11 Complexity: 75 Confidence: 88 Actionability: 65

Issue body

TL;DR: as of python 3.7, .pyc files by default include a timestamp and a size of the source file which results in a change of a hash every time a .pyc file is generated for a given source file. This results in staging conflicts for python parts. https://docs.python.org/3/library/py_compile.html#py_compile.compile https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode.TIMESTAMP Even if this is set, there is still an issue https://github.com/pypa/pip/issues/8414 Description/analysis: When building a project with multiple identical python dependencies I consistently get an error like this: Failed to stage: Parts 'openstack-projects' and 'cluster' have the following files, but with different contents: bin/activate bin/activate.csh bin/activate.fish bin/python3 pyvenv.cfg bin/python3 lib/python3.8/site-packages/Flask-1.1.2.dist-info/RECORD lib/python3.8/site-packages/__pycache__/easy_install.cpython-38.pyc lib/python3.8/site-packages/certifi/__pycache__/__init__.cpython-38.pyc lib/python3.8/site-packages/certifi/__pycache__/__main__.cpython-38.pyc # many other .pyc files ... While snapcraft suggests that I use something like `organize`, `filesets` and `stage`, the issue is that the source files for those dependencies are identical - there is no reason for any manual work here. Source hashes are the same: snapcraft-microstack # sha256sum ./parts/cluster/install/lib/python3.8/site-packages/click/_textwrap.py 6a30b3933165cb9b639bd7e843937dfcc39e69824c063025b6e15aebd9f88976 ./parts/cluster/install/lib/python3.8/site-packages/click/_textwrap.py snapcraft-microstack # sha256sum ./parts/openstack-projects/install/lib/python3.8/site-packages/click/_textwrap.py 6a30b3933165cb9b639bd7e843937dfcc39e69824c063025b6e15aebd9f88976 ./parts/openstack-projects/install/lib/python3.8/site-packages/click/_textwrap.py .pyc files are different: snapcraft-microstack # sha256sum ./parts/openstack-projects/install/lib/python3.8/site-packages/click/__pycache__/_textwrap.cpython-38.pyc 398b47a5abfc87e9da73153e42d48dcd5d917bd637a0e0af1eb6999f19fb1085 ./parts/openstack-projects/install/lib/python3.8/site-packages/click/__pycache__/_textwrap.cpython-38.pyc snapcraft-microstack # sha256sum ./parts/cluster/install/lib/python3.8/site-packages/click/__pycache__/_textwrap.cpython-38.pyc d4642cfecd727d228944a1d31ff728e7ef6529a7a88898f6568ea6e96d1f8f82 ./parts/cluster/install/lib/python3.8/site-packages/click/__pycache__/_textwrap.cpython-38.pyc RECORD files include hashes as well, hence they are also different: snapcraft-microstack # diff ./parts/openstack-projects/install/lib/python3.8/site-packages/Flask-1.1.2.dist-info/RECORD ./parts/cluster/install/lib/python3.8/site-packages/Flask-1.1.2.dist-info/RECORD 1c1 < ../../../bin/flask,sha256=VXQqccMeG03Rn8_yN8Kq3Up13rzyaoHsEckFnCxHor4,242 --- > ../../../bin/flask,sha256=NAzPpe84iZFX3PYsCZEirt3fAFObAjBuCpM25792kSU,231 Apparently, as of python 3.7, .pyc files include a timestamp and a size of the source by default (PycInvalidationMode.TIMESTAMP). There is a way to override this behavior by setting the SOURCE_DATE_EPOCH environment variable to switch py_compile to using PycInvalidationMode.CHECKED_HASH: https://docs.python.org/3/library/py_compile.html py_compile.compile(file, cfile=None, dfile=None, doraise=False, optimize=-1, invalidation_mode=PycInvalidationMode.TIMESTAMP, quiet=0) invalidation_mode should be a member of the PycInvalidationMode enum and controls how the generated bytecode cache is invalidated at runtime. The default is PycInvalidationMode.CHECKED_HASH ***if the SOURCE_DATE_EPOCH environment variable is set***, otherwise ***the default is PycInvalidationMode.TIMESTAMP***. https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode.TIMESTAMP TIMESTAMP The .pyc file includes the timestamp and size of the source file, which Python will compare against the metadata of the source file at runtime to determine if the .pyc file needs to be regenerated. https://docs.python.org/3/library/py_compile.html#py_compile.PycInvalidationMode.CHECKED_HASH CHECKED_HASH The .pyc file includes a hash of the source file content, which Python will compare against the source at runtime to determine if the .pyc file needs to be regenerated. Adding something like this seems to be needed: build-environment: - SOURCE_DATE_EPOCH: '1591640328' However, see https://bugs.launchpad.net/snapcraft/+bug/1882535/comments/2

Evaluation history

Date	Model	Scores	Action	Summary
2026-10-02 06:30 PM UTC	stealth/space-bunny-alpha	Impact: 65 Quick Win: 11 Complexity: 75 Confidence: 88 Actionability: 65	keep open	Overlapping Python dependencies generate nondeterministic bytecode, RECORD, and venv artifacts that block staging; confirmed to affect multiple users, with workarounds but no complete fix.
2026-10-02 06:30 PM UTC	stealth/space-bunny-alpha	Impact: 64 Quick Win: 14 Complexity: 72 Confidence: 93 Actionability: 79	keep open	Confirmed staging failure when Python parts install identical dependencies: nondeterministic .pyc, RECORD, and venv files collide. Workarounds exist, but the report remains unresolved.
2026-09-30 08:07 PM UTC	stealth/space-bunny-alpha	Impact: 56 Quick Win: 16 Complexity: 62 Confidence: 89 Actionability: 74	keep open	Identical Python dependencies produce nondeterministic .pyc, RECORD, and venv files that block staging; confirmed with workarounds but no verified complete fix.
2026-09-22 08:53 PM UTC	qwen/qwen3.6-35b-a3b	Impact: 70 Quick Win: 39 Complexity: 30 Confidence: 60 Actionability: 80	keep open	Snapcraft staging conflicts occur when multiple Python parts share dependencies due to non-deterministic .pyc file generation in Python 3.7+. Issue is open, unlabelled, and lacks maintainer response.
2026-09-01 06:37 AM UTC	qwen/qwen3.8-27b	Impact: 40 Quick Win: 18 Staleness: 90 Complexity: 55 Confidence: 60 Support Request: 15	needs triage	Staging conflicts when multiple python parts share identical dependencies: Python 3.7+ .pyc files embed source timestamps, so identical sources produce different .pyc/RECORD hashes across parts. Reported for focal/core20, no maintainer response, 0 comments, open ~6 years.
2026-07-23 11:36 AM UTC	qwen/qwen3.6-35b-a3b	Staleness: 98 Complexity: 45 Confidence: 90 Support Request: 15	close stale	Staging conflicts arise when multiple Python parts share dependencies due to differing .pyc hashes from Python 3.7+ timestamp behavior. Reported six years ago with no maintainer response, targeting outdated Ubuntu releases.
2026-07-22 01:24 PM UTC	qwen/qwen3.6-35b-a3b	Staleness: 95 Complexity: 45 Confidence: 85 Support Request: 15	close stale	Snapcraft staging conflicts occur when multiple Python parts share dependencies due to Python 3.7+ .pyc timestamp hashing. Untriaged, inactive for ~2.7 years, no maintainer response.
2026-07-09 12:39 PM UTC	qwen3.6-35b-a3b-mtp-q6	Staleness: 95 Complexity: 65 Confidence: 85 Support Request: 5	close stale	Snapcraft staging conflicts occur when multiple Python parts share identical dependencies due to differing .pyc file hashes in Python 3.7+. Issue is unlabelled, has zero comments, and has been inactive for over six years.

Update history

Date	Change
2023-11-11 03:03 AM UTC	created

Related work

Blocked By: pip#8414 (confidence 99%)
The issue explicitly identifies this upstream pip behavior as preventing SOURCE_DATE_EPOCH from producing identical bytecode across runs.
Related To: snapcraft (launchpad)#2080356 (confidence 98%)
A later core22 report describes overlapping Python parts and setuptools staging conflicts.
Related To: snapcraft (launchpad)#1502105 (confidence 90%)
An earlier report covers staging conflicts caused by differing .pyc files from two Python 3 parts.

Related issues

Issue	Project	State	Summary	Similarity
#2080356 Two python3 parts using setuptools are always conflicting on core22	snapcraft (launchpad)	open	Two python plugin parts always collide on core22 because each part's venv embeds its own install path in bin/activate*, __pycache__ and RECORD; only workaround is manual stage exclusions. No maintainer response.	71%