Thanks to visit codestin.com
Credit goes to github.com

Skip to content

ZipFile.writestr should respect SOURCE_DATE_EPOCH #91279

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
SomberNight mannequin opened this issue Mar 25, 2022 · 1 comment
Closed

ZipFile.writestr should respect SOURCE_DATE_EPOCH #91279

SomberNight mannequin opened this issue Mar 25, 2022 · 1 comment
Labels
3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement

Comments

@SomberNight
Copy link
Mannequin

SomberNight mannequin commented Mar 25, 2022 •
edited by bedevere-app bot
Loading

BPO 47123
Nosy @SomberNight
Files
  • zipfile_respect_sourcedate.diff: change the ZipFile.writestr to respect SOURCE_DATE_EPOCH
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2022-03-25.19:47:10.113>
labels = ['type-feature', 'library', '3.9', '3.10', '3.11']
title = 'ZipFile.writestr should respect SOURCE_DATE_EPOCH'
updated_at = <Date 2022-03-25.19:47:10.113>
user = 'https://github.com/SomberNight'

    bugs.python.org fields:

    activity = <Date 2022-03-25.19:47:10.113>
actor = 'ghost43'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)']
creation = <Date 2022-03-25.19:47:10.113>
creator = 'ghost43'
dependencies = []
files = ['50702']
hgrepos = []
issue_num = 47123
keywords = ['patch']
message_count = 1.0
messages = ['416015']
nosy_count = 1.0
nosy_names = ['ghost43']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue47123'
versions = ['Python 3.9', 'Python 3.10', 'Python 3.11']

    Linked PRs
    @SomberNight
    Copy link
    Mannequin Author

    SomberNight mannequin commented Mar 25, 2022

    Currently ZipFile.writestr writes the local time into the ZipFile.
    (depends on both current time and local timezone)
    See

    cpython/Lib/zipfile.py

    Lines 1816 to 1817 in 20e6e56

    zinfo = ZipInfo(filename=zinfo_or_arcname,
    date_time=time.localtime(time.time())[:6])

    This makes pip installing a package generate non-reproducible build artifacts.

    Specifically, Scripts/*.exe files (created for packages that define entry_points/console_scripts) are not reproducible on Windows when installed by pip. This also leaks into the *.dist-info/RECORD files.

    For example, after running pip install wheel or pip install pyinstaller,
    in wheel-0.37.1.dist-info/RECORD, I have this line:

    ../../Scripts/wheel.exe,sha256=u9TbPw2XNs_F9uy7y2zwumuzAZDbOSB7BXjLHZ0tTHg,97103

    in pyinstaller-4.10.dist-info/RECORD, I have these lines:

    ../../Scripts/pyi-archive_viewer.exe,sha256=nC-9muPlIhUC1qvFkXHpyKJyRQqXISXxbUPXQ1XVOiM,97133
../../Scripts/pyi-bindepend.exe,sha256=udFHiAdndPpSwaIqmhmLEy36IUs1cNNoNQznSEnLJQQ,97128
../../Scripts/pyi-grab_version.exe,sha256=3ET9E841tFWujFL99aG4frzgwlP9f9pAkMgE0k2UGK0,97131
../../Scripts/pyi-makespec.exe,sha256=dJkfmITdLJhyPngmqziqqj5tH9qqfeQc5BTubeoXWUs,97127
../../Scripts/pyi-set_version.exe,sha256=sWmcOVS93fUY-wbdoz6ixBCvjy1tC4Aaw30DMmrmo-0,97130
../../Scripts/pyinstaller.exe,sha256=haInbhH0pImJn24cW4v917oUZmzXZj8OE89KFh4MO2Y,97112

    Upon comparing multiple Scripts/wheel.exe files, I've found that the only difference is due to the above-mentioned timestamp embedded inside the exe (or rather, same timestamp embedded twice).

    The exe files get created by distlib (vendored by pip).
    Here is a traceback with an artificial exception to illustrate the codepath:

    (env) PS C:\tmp> pip install --no-build-isolation pyinstaller
Collecting pyinstaller
  Using cached pyinstaller-4.10-py3-none-win_amd64.whl (2.0 MB)
Requirement already satisfied: setuptools in c:\tmp\env\lib\site-packages (from pyinstaller) (61.0.0)
Requirement already satisfied: pyinstaller-hooks-contrib>=2020.6 in c:\tmp\env\lib\site-packages (from pyinstaller) (2022.3)
Requirement already satisfied: altgraph in c:\tmp\env\lib\site-packages (from pyinstaller) (0.17.2)
Requirement already satisfied: pefile>=2017.8.1 in c:\tmp\env\lib\site-packages (from pyinstaller) (2021.9.3)
Requirement already satisfied: pywin32-ctypes>=0.2.0 in c:\tmp\env\lib\site-packages (from pyinstaller) (0.2.0)
Requirement already satisfied: future in c:\tmp\env\lib\site-packages (from pefile>=2017.8.1->pyinstaller) (0.18.2)
Installing collected packages: pyinstaller
ERROR: Exception:
Traceback (most recent call last):
  File "C:\tmp\env\lib\site-packages\pip\_internal\cli\base_command.py", line 167, in exc_logging_wrapper
    status = run_func(*args)
  File "C:\tmp\env\lib\site-packages\pip\_internal\cli\req_command.py", line 205, in wrapper
    return func(self, options, args)
  File "C:\tmp\env\lib\site-packages\pip\_internal\commands\install.py", line 405, in run
    installed = install_given_reqs(
  File "C:\tmp\env\lib\site-packages\pip\_internal\req\__init__.py", line 73, in install_given_reqs
    requirement.install(
  File "C:\tmp\env\lib\site-packages\pip\_internal\req\req_install.py", line 769, in install
    install_wheel(
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 729, in install_wheel
    _install_wheel(
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 646, in _install_wheel
    generated_console_scripts = maker.make_multiple(scripts_to_generate)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 440, in make_multiple
    filenames.extend(self.make(specification, options))
  File "C:\tmp\env\lib\site-packages\pip\_internal\operations\install\wheel.py", line 427, in make
    return super().make(specification, options)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 429, in make
    self._make_script(entry, filenames, options=options)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 329, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\tmp\env\lib\site-packages\pip\_vendor\distlib\scripts.py", line 263, in _write_script
    raise Exception(f"heyheyhey2. {sha256(launcher)=}. {sha256(shebang)=}. {sha256(zip_data)=}. " +
Exception: heyheyhey2. sha256(launcher)='a00a877acefc'. sha256(shebang)='58628e924f22'. sha256(zip_data)='a423496a0482'. ('SOURCE_DATE_EPOCH' in os.environ)=True

    The interesting code is here:
    https://github.com/pypa/distlib/blob/d0e3f49df5d1aeb9daeaaabf0391c9e13e4a6562/distlib/scripts.py#L251-L252
    This calls into the cpython standard library, where time.time() gets written into the file:

    cpython/Lib/zipfile.py

    Lines 1816 to 1817 in 20e6e56

    zinfo = ZipInfo(filename=zinfo_or_arcname,
    date_time=time.localtime(time.time())[:6])

    Ideally, either distlib or the stdlib zipfile module should be changed to respect SOURCE_DATE_EPOCH, but it's not entirely clear to me which...

    The attached patch changes ZipFile.writestr to respect SOURCE_DATE_EPOCH if set.

    related pypa/distlib#164
    related spesmilo/electrum#7739

    @SomberNight SomberNight mannequin added 3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement labels Mar 25, 2022
    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @Wulian233 Wulian233 mentioned this issue Sep 24, 2024
    Merged
    jaraco pushed a commit that referenced this issue Jan 20, 2025
    @Wulian233
    gh-91279: ZipFile.writestr now respect SOURCE_DATE_EPOCH (#124435)
    5d57959
    srinivasreddy pushed a commit to srinivasreddy/cpython that referenced this issue Jan 21, 2025
    @Wulian233 @srinivasreddy
    pythongh-91279: ZipFile.writestr now respect SOURCE_DATE_EPOCH (pytho…
    a3b40da 
    …n#124435)
    @Eclips4 Eclips4 closed this as completed Jan 22, 2025
    @ctrlaltf2 ctrlaltf2 mentioned this issue May 19, 2025
    Open
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    3.9 only security fixes 3.10 only security fixes 3.11 only security fixes stdlib Python modules in the Lib dir type-feature A feature request or enhancement
    Projects
    Zipfile issues
    Status: Done
    Milestone
    No milestone
    Development

    No branches or pull requests
    1 participant
    @Eclips4