Thanks to visit codestin.com
Credit goes to github.com

Skip to content

State of os.path and path types #1841

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
FichteFoll opened this issue Jan 24, 2018 · 11 comments
Closed

State of os.path and path types #1841

FichteFoll opened this issue Jan 24, 2018 · 11 comments

Comments

@FichteFoll
Copy link
Contributor

FichteFoll commented Jan 24, 2018

According to a few older PRs (#1150, #1459), it seems like the stubs for for 2 and 3 should be merged into a 2and3 version, however some commits continue to introduce changes to only one of the os files (notably #1645). Merging them is supposedly blocked on os/__init__.py.

Is this merge still a goal moving forward? If not, I suppose they would benefit from some simplification (notably the 2 version).

Furthermore, I intend to add a couple more _PathType type signatures to os.path functions as pretty much all of them support it internally, but only ~half of them are annotated correctly. Ideally, Union[AnyStr, _PathLike[AnyStr]] is to be used for the _PathType type (via #1441), but it seems like that is still blocked on python/mypy#3644. When replacing AnyStr with _PathType (with _PathLike instead of _PathLike[AnyStr]), the generic properties of the functions signatures are lost, so that would be a regression, but the annotations are currently in a mixed state anyway.

@gvanrossum
Copy link
Member

gvanrossum commented Jan 24, 2018 via email

@FichteFoll
Copy link
Contributor Author

All right. I don't think I have the capacity currently to merge the os/__init__.pyi stubs, but I'd be interested in the second part, as I ran into that when passing a pathlib.Path object to os.path.relpath.

To formulate that into a question: Should I replace signatures in the form def x(a: AnyStr) -> AnyStr: ... with def x(a: _PathType) -> AnyStr: ..., which loses the generic property of returning the same string type as passed but gains support for PathLike objects? That is, until python/mypy#3644 is fixed.

@gvanrossum
Copy link
Member

Maybe you could do something like what was done for readlink()?

if sys.version_info >= (3, 6):
    def readlink(path: Union[AnyStr, PathLike[AnyStr]], *, dir_fd: Optional[int] = ...) -> AnyStr: ...

@FichteFoll
Copy link
Contributor Author

Isn't that exactly what triggers python/mypy#3644? Or would that only apply to _PathType[AnyStr]? In that case, I could as well just change _PathType = Union[AnyStr, _PathLike[AnyStr]]. Or, if that is undesireable, add a different alias for that.

@gvanrossum
Copy link
Member

That's an interesting option. I'm too tired tonight to consider it, sorry. Maybe @JukkaL has an opinion. At the very least you should check everything that currently uses _PathType and see if/whether it would (or should!) change.

@JukkaL
Copy link
Contributor

JukkaL commented Jan 24, 2018

One option is to use an overloaded function, which mypy seems to handle already:

@overload
def readlink(path: AnyStr, *, dir_fd: Optional[int] = 0) -> AnyStr: ...
@overload
def readlink(path: PathLike[AnyStr], *, dir_fd: Optional[int] = 0) -> AnyStr: ...

Or we could prioritize fixing python/mypy#3644 -- it doesn't feel to me like it would be a tricky one, but I could be mistaken.

@FichteFoll
Copy link
Contributor Author

For starters, I tried replacing all current usages of AnyStr in parameters with _GenPathType (short for generic) and the following:

if sys.version_info >= (3, 6):
    from builtins import _PathLike
    _PathType = Union[bytes, Text, _PathLike]
    _GenPathType = Union[AnyStr, _PathLike[AnyStr]]
else:
    _PathType = Union[bytes, Text]
    _GenPathType = AnyStr

However, running mypy_test.py, all Python 3 versions <3.6 complain about

stdlib/3/os/path.pyi:20: error: Type variable "typing.AnyStr" is invalid as target for type alias
stdlib/3/os/path.pyi:35: error: Invalid type "os.path._GenPathType"
…

@gvanrossum
Copy link
Member

Looks like you can't have an alias for a type variable (like AnyStr).

Another option to look into might be to define _PathLike and _PathType unconditionally, and only do the export of _PathType as PathType conditionally.

@FichteFoll
Copy link
Contributor Author

Do the export where?

@gvanrossum
Copy link
Member

gvanrossum commented Jan 27, 2018 via email

@JelleZijlstra
Copy link
Member

The parts about PathType are covered in the more focused #1997, and the question about merging has been answered, so closing this issue.

@FichteFoll FichteFoll changed the title State of os.path State of os.path and path types Apr 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants