gh-51067: add `ZipFile.remove()` #103033

danny0838 · 2023-03-25T11:30:03Z

This is a revision of #19358 (for issue #51067) as the original author seems not keeping working.

Notable changes:

Added docs and tests.
Support mode 'w' and 'x', as noted by remove/delete method for zipfile objects #51067 (comment)
Support removing multiple members and removing non-physically with the internal _remove_members method, as they may be used by some interested people, as noted by remove/delete method for zipfile objects #51067 (comment) and remove/delete method for zipfile objects #51067 (comment).

They are not currently introduced in the public remove API, as it would involve more complicated changes to the public APIs (e.g. introducing error handling for multiple members, and a extra method that purges stale data by non-physical removing) and other ZipFile related APIs do not support similar operations.
Move physical data in chunks, to prevent a memory issue for large files.
Fixed a flaw of the previous implementation that self.NameToInfo gets a missing key when removing one of duplicated arcnames.

Issue: remove/delete method for zipfile objects #51067

📚 Documentation preview 📚: https://cpython-previews--103033.org.readthedocs.build/

ghost · 2023-03-25T11:30:06Z

All commit authors signed the Contributor License Agreement.

bedevere-bot · 2023-03-25T11:30:07Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

bedevere-bot · 2023-03-25T11:31:15Z

Most changes to Python require a NEWS entry.

Please add it using the blurb_it web app or the blurb command-line tool.

barneygale · 2023-03-30T17:58:51Z

Automatically compacting the .zip file seems like overkill. Suggestion: split this into two methods:

ZipInfo.remove() removes the record for the file, and by default zeroes out its data.
ZipFile.repack() reclaims free space.

The latter method is dangerous for self-extracting .zip files, which have an executable header before the the zip data begins. That header would be stripped out, I think.

barneygale · 2023-03-30T17:59:27Z

Also, please don't force-push to an open PR. It makes it harder for reviewers to follow changes! Thanks

Refer to: python/cpython#103033

* Add remove method to ZipFile Refer to: python/cpython#103033 * Make use of `ZipFileWithRemove`

Doc/library/zipfile.rst

Lib/zipfile/__init__.py

python-cla-bot · 2025-05-22T12:37:02Z

All commit authors signed the Contributor License Agreement.

bedevere-app · 2025-05-22T12:37:03Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

This is a revision of commit 659eb04 (PR python#19358), notably with following changes: - Add documentation and tests. - Raise `ValueError` for a bad mode, as in other methods. - Support multi-member removal in `_remove_members()`. - Support non-physical removal in `_remove_members()`. - Move physical file data in chunks to prevent excessive memory usage on large files. - Fix missing entry in `self.NameToInfo` when removing a duplicated archive name. - Also update `ZipInfo._end_offset` for physically moved files. Co-authored-by: Éric <[email protected]> (cherry picked from commit e6bc82a (PR python#103033))

bedevere-app · 2025-05-22T12:45:30Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

This is a revision of commit 659eb04 (PR python#19358), notably with following changes: - Add documentation and tests. - Raise `ValueError` for a bad mode, as in other methods. - Support multi-member removal in `_remove_members()`. - Support non-physical removal in `_remove_members()`. - Move physical file data in chunks to prevent excessive memory usage on large files. - Fix missing entry in `self.NameToInfo` when removing a duplicated archive name. - Also update `ZipInfo._end_offset` for physically moved files. Co-authored-by: Éric <[email protected]> (cherry picked from commit e6bc82a (PR python#103033))

bedevere-app · 2025-05-22T12:51:43Z

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

danny0838 · 2025-05-22T12:59:50Z

The PR is being old and the base python executable can be hardly compiled. Rebased onto the latest Python and make sure it builds, with few trivial commits squashed together.

merwok · 2025-06-19T14:52:03Z

Misc/NEWS.d/next/Core_and_Builtins/2025-05-22-12-52-35.gh-issue-51067.tJxGGF.rst

@@ -0,0 +1 @@
+Add ``ZipFile.remove()``


Suggested change

Add ``ZipFile.remove()``

Add :meth:`ZipFile.remove <zipfile.ZipFile.remove>`.

This is a revision of commit 659eb04 (PR python#19358), notably with following changes: - Add documentation and tests. - Raise `ValueError` for a bad mode, as in other methods. - Support multi-member removal in `_remove_members()`. - Support non-physical removal in `_remove_members()`. - Move physical file data in chunks to prevent excessive memory usage on large files. - Fix missing entry in `self.NameToInfo` when removing a duplicated archive name. - Also update `ZipInfo._end_offset` for physically moved files. Co-authored-by: Éric <[email protected]> (cherry picked from commit e6bc82a (PR python#103033))

- File is not truncated in mode 'w'/'x', which results non-shrinked file. - This cannot be simply resolved by adding truncation for mode 'w'/'x', which may be used on an unseekable file buffer and truncation is not allowed.

- The seek will be automatically called in `ZipFile.close`.

danny0838 · 2025-06-19T16:16:08Z

Rebased to use the same base as PR #134627 for easier comparison.

In favor of #134627, I'm probably not going to keep working on this PR, unless it proves to be the final accepted approach for the issue.

NOTE: Compared to #134627, this PR is rather quick and naive, and some cares may need to be aware of:

Overlapping entries are not checked. It may be unsafe to call remove on a ZIP file having such entries.
When stripping a local file entry, the algorithm simply strips bytes between its starting and next local file entry's starting instead of actually calculate its size. This means extra bytes after the entry will be removed, which may be a problem in some cases. (See c80d21b for details)

Unfortunately, fixing any of them requires a non-trivial code rework.

bedevere-bot mentioned this pull request Mar 25, 2023

remove/delete method for zipfile objects #51067

Open

bedevere-bot added the awaiting review label Mar 25, 2023

danny0838 force-pushed the gh-51067 branch from 2219d90 to 314916e Compare March 25, 2023 11:31

danny0838 force-pushed the gh-51067 branch 3 times, most recently from 76722fa to f3450f1 Compare March 26, 2023 12:37

arhadthedev added the stdlib Python modules in the Lib dir label Mar 30, 2023

bpepple added a commit to Metron-Project/darkseid that referenced this pull request Aug 31, 2024

Add remove method to ZipFile

343c935

Refer to: python/cpython#103033

bpepple added a commit to Metron-Project/darkseid that referenced this pull request Aug 31, 2024

Add remove method to ZipFile (#111)

c779b46

* Add remove method to ZipFile Refer to: python/cpython#103033 * Make use of `ZipFileWithRemove`

merwok reviewed Apr 24, 2025

View reviewed changes

Doc/library/zipfile.rst Outdated Show resolved Hide resolved

Lib/zipfile/__init__.py Show resolved Hide resolved

Lib/zipfile/__init__.py Outdated Show resolved Hide resolved

merwok added the type-feature A feature request or enhancement label Apr 24, 2025

danny0838 force-pushed the gh-51067 branch from 692260e to 4556a74 Compare May 22, 2025 12:36

danny0838 force-pushed the gh-51067 branch from 4556a74 to 7632c7b Compare May 22, 2025 12:45

danny0838 force-pushed the gh-51067 branch from a43b29f to f5325ad Compare May 22, 2025 12:51

danny0838 changed the title ~~gh-51067: add ZipInfo.remove()~~ gh-51067: add ZipFile.remove() May 22, 2025

danny0838 mentioned this pull request May 24, 2025

gh-51067: Add remove() and repack() to ZipFile #134627

Open

merwok reviewed Jun 19, 2025

View reviewed changes

danny0838 and others added 10 commits June 19, 2025 23:44

Fix and improve tests

96a07a2

Update ZipInfo._end_offset

602b909

No more allow mode 'w'/'x'

bb90f48

- File is not truncated in mode 'w'/'x', which results non-shrinked file. - This cannot be simply resolved by adding truncation for mode 'w'/'x', which may be used on an unseekable file buffer and truncation is not allowed.

Test for zip64 more thoroughly

2d74501

Add tests for zstd

63d56e5

Remove unneeded seek

ab3a187

- The seek will be automatically called in `ZipFile.close`.

Add lock for thread safety

1c074c5

📜🤖 Added by blurb_it.

8b4f709

Suppress duplicated name warning during generating test files

7b7f54a

Fix rst format

c95ef53

danny0838 force-pushed the gh-51067 branch from 0d82d00 to c95ef53 Compare June 19, 2025 15:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

gh-51067: add `ZipFile.remove()` #103033

gh-51067: add `ZipFile.remove()` #103033

danny0838 commented Mar 25, 2023 •

edited

Loading

Uh oh!

ghost commented Mar 25, 2023 •

edited by ghost

Loading

Uh oh!

bedevere-bot commented Mar 25, 2023

Uh oh!

bedevere-bot commented Mar 25, 2023

Uh oh!

barneygale commented Mar 30, 2023

Uh oh!

barneygale commented Mar 30, 2023 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

python-cla-bot bot commented May 22, 2025 •

edited

Loading

Uh oh!

bedevere-app bot commented May 22, 2025

Uh oh!

bedevere-app bot commented May 22, 2025

Uh oh!

bedevere-app bot commented May 22, 2025

Uh oh!

danny0838 commented May 22, 2025

Uh oh!

merwok Jun 19, 2025

Uh oh!

danny0838 commented Jun 19, 2025 •

edited

Loading

Uh oh!

Uh oh!

	Add ``ZipFile.remove()``
	Add :meth:`ZipFile.remove <zipfile.ZipFile.remove>`.

Uh oh!

gh-51067: add ZipFile.remove() #103033

Are you sure you want to change the base?

gh-51067: add ZipFile.remove() #103033

Conversation

danny0838 commented Mar 25, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Mar 25, 2023 • edited by ghost Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-bot commented Mar 25, 2023

Uh oh!

bedevere-bot commented Mar 25, 2023

Uh oh!

barneygale commented Mar 30, 2023

Uh oh!

barneygale commented Mar 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

python-cla-bot bot commented May 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app bot commented May 22, 2025

Uh oh!

bedevere-app bot commented May 22, 2025

Uh oh!

bedevere-app bot commented May 22, 2025

Uh oh!

danny0838 commented May 22, 2025

Uh oh!

merwok Jun 19, 2025

Choose a reason for hiding this comment

Uh oh!

danny0838 commented Jun 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

gh-51067: add `ZipFile.remove()` #103033

gh-51067: add `ZipFile.remove()` #103033

danny0838 commented Mar 25, 2023 •

edited

Loading

ghost commented Mar 25, 2023 •

edited by ghost

Loading

barneygale commented Mar 30, 2023 •

edited

Loading

python-cla-bot bot commented May 22, 2025 •

edited

Loading

danny0838 commented Jun 19, 2025 •

edited

Loading