Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

ap--
Copy link
Collaborator

@ap-- ap-- commented Mar 16, 2025

Draft PR for starting implementation for fsspec urlpath chain support.

Closes #28

  • UPath should start relying on CurrentChainSegment to store path, protocol, and storage_options.
  • Need to think through if by default, the target protocol should be the first or last protocol used in the chain
  • There needs to be some way to select which chain segment UPath will operate on
    >>> p0 = UPath("zip://abc/efg.csv::s3://bucket/file.zip")
    >>> p0
    S3Path("zip://abc/efg.csv::s3://bucket/file.zip")
    >>> p1 = p0.chain[-2]  # maybe this?
    >>> p1 = p0.chain_target("zip")  # or maybe this?
    >>> p1
    ZipPath("zip://abc/efg.csv::s3://bucket/file.zip")
  • We need to figure out how filesystems that just pass the path through show up. For example SimpleCache path semantics should adhere to the downstream path semantics... (ChainSegment.path can be None for detecting these cases)
  • Add tests for chaining behaviour
  • Fix .parts regression since v0.2.6

@ap-- ap-- marked this pull request as draft March 16, 2025 08:51
@emfdavid emfdavid mentioned this pull request Mar 18, 2025
@ap-- ap-- added this to the v0.3.0 milestone May 8, 2025
@ap--
Copy link
Collaborator Author

ap-- commented May 29, 2025

Almost there.
Needs more tests and an interface to select the correct chain element.

segments might be intuitive when talking about a urlpath chain but I think stdlib pathlib might use the term in the future, so it should be something more fsspec specific.

@ap-- ap-- marked this pull request as ready for review September 5, 2025 21:24
@ap--
Copy link
Collaborator Author

ap-- commented Sep 5, 2025

Merging this now, and will track exposing the segment selection in another PR.

@ap-- ap-- merged commit ed559a0 into fsspec:main Sep 5, 2025
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

fsspec URL chaining
1 participant