Thanks to visit codestin.com
Credit goes to github.com

Skip to content

zipfile.Path has only partial compatibility io.BytesIO #130120

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
yngvem opened this issue Feb 14, 2025 · 2 comments
Closed

zipfile.Path has only partial compatibility io.BytesIO #130120

yngvem opened this issue Feb 14, 2025 · 2 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@yngvem
Copy link
Contributor

yngvem commented Feb 14, 2025

Bug description:

I encountered an edge-case while using zipfile.Path with in-memory data buffers. It seems like the filename of the root is set to None, which breaks anything that tries to join the zip-filename and its internal path (including str(), ``.

I have two suggested fixes:

  1. Update the initialization of the zipfile.FastLookup-class and set self.filename to ":memory:" if it is None, matching what you would write in e.g. sqlite to get an in-memory database.
  2. Update all places where the code attempts do Path-operations on self.root.filename to notice if the filename is None and use ":memory:" instead.

The first is definitely the easiest fix, but the second would maybe be more backwards compatible in case someone uses Path.root.filename to notice that the zipfile.Path-object points to an in-memory zipfile. Though, I'm not sure how big of a problem that is.

I don't have any strong opinions of which to choose, and I would be happy to implement the fix.

Minimal reproducible example
import io
import zipfile

# Create a dummy Zip file
data = io.BytesIO()
with zipfile.ZipFile(data, mode="w") as zf:
    zf.writestr("hello.txt", "python\n")

# Try to iterate over its content
data.seek(0)
p = zipfile.Path(data)
print(str(p))
print(list(p.iterdir()))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[92], line 13
     11 p = zipfile.Path(data)
---> 12 print(str(p))

File /usr/lib/python3.12/zipfile/_path/__init__.py:391, in Path.__str__(self)
    390 def __str__(self):
--> 391     return posixpath.join(self.root.filename, self.at)

File <frozen posixpath>:76, in join(a, *p)

TypeError: expected str, bytes or os.PathLike object, not NoneType
Demonstration of fix
import io
import zipfile

# Create a dummy Zip file
data = io.BytesIO()
with zipfile.ZipFile(data, mode="w") as zf:
    zf.writestr("hello.txt", "python\n")

# Try to iterate over its content
data.seek(0)
p = zipfile.Path(data)
p.root.filename = ":memory:"
print(str(p))
print(list(p.iterdir()))
:memory:/
[Path(':memory:', 'hello.txt')]

CPython versions tested on:

3.12, 3.14

Operating systems tested on:

Ubuntu 24.04 (WSL), Fedora (via the Python devcontainer on GitHub codespaces)

Linked PRs

@yngvem yngvem added the type-bug An unexpected behavior, bug, or error label Feb 14, 2025
@yngvem
Copy link
Contributor Author

yngvem commented Feb 14, 2025

I just realised that option 2. might be preferable, because we should probably overload the parent method to prevent it from resolving to "." when the path is ":memory:" anyways?

@picnixz picnixz added the stdlib Python modules in the Lib dir label Feb 14, 2025
@yngvem
Copy link
Contributor Author

yngvem commented Feb 20, 2025

I've started working on this now. I had one idea though: How about setting the name to \0 if it represents an in-memory byte string? That way it's clear that it isn't a local path with the name ":memory:".

Update: Looking at the tests, it seems like it's supposed to fail. So I'll only update the code so __str__ doesn't fail

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Status: Done
Development

No branches or pull requests

3 participants