Improve type annotations and type correctness #191
Conversation
| self._absolute_path_validation() | ||
| if not self.key: | ||
| return None | ||
| raise ValueError('Key is required for stat') |
There was a problem hiding this comment.
This is a behavior change, but the upstream Path.stat does not ever return None and the existing type annotation (before this PR) on S3Path.stat indicates that this shouldn't return None.
I think the exception is also the most appropriate behavior, as attempting to call S3Path.stat on something without a key makes no sense (unless you wanted bucket-level stats, which would probably have a totally different return type).
There was a problem hiding this comment.
very good point
We should add this behavior change to the doc's as well
Tx
| raise ValueError("Absolute path can't be determined for relative S3Path objects") | ||
|
|
||
| def owner(self) -> str: | ||
| def owner(self, *, follow_symlinks: bool = True) -> str: |
There was a problem hiding this comment.
follow_symlinks was added in Python 3.13 to Path.is_file, Path.is_dir, Path.owner(), and Path.group(). This change makes the signatures compatible with Python 3.13+, but since they're optional keyword-only parameters, they shouldn't disrupt compatibility with older Python versions.
Note, other places attempt to defensively program around a user who tries to set follow_symlinks, but while I didn't remove those checks, I think they're excessive. It's fine to have arguments which are simply ignored.
There was a problem hiding this comment.
OK
Let's add follow_symlinks as default False - so we won't have behavior change
And add validation because follow_symlinks is not supported in S3
| """ | ||
| self._absolute_path_validation() | ||
| if not isinstance(other_path, Path): | ||
| if not isinstance(other_path, S3Path): |
There was a problem hiding this comment.
This is a behavior change, but I think it's actually a bugfix.
| bucket_path = cls(cls.parser.sep, bucket) | ||
| if len(bucket_path.parts) != 2: | ||
| raise ValueError(f'bucket argument contains more then one path element: {bucket_path}') | ||
| key_path = cls(key) | ||
| if key_path.is_absolute(): | ||
| key_path = key_path.relative_to('/') | ||
| return bucket_path / key_path |
There was a problem hiding this comment.
This doesn't change any behavior, it just uses a new variable to prevent type confusion from overwriting a parameter with a different type.
There was a problem hiding this comment.
instead of adding the bucket_path,
What do you thing about changing the method hints?
def from_bucket_key(cls, bucket: Union[str, os.PathLike[str]], key: Union[str, os.PathLike[str]]) -> Self:now that I see this class method it make more sense no?
This method can get path object as well
This adds a number of missing type annotations to the public API surface. It also fixes the signature of several methods which allow a `follow_symlinks` parameter in Python 3.13+. This parameter is allowed in `S3Path`, but ignored. Unfortunately, these changes are not enough for MyPy to run without errors.
There was a problem hiding this comment.
Hi @brianhelba,
First, I’d like to thank you for your contribution and support for s3path.
Regarding type hints—I really like them when they help clean up the code and improve visibility, especially when debugging or trying to understand the logic. However, I also value Python’s simplicity and clean syntax.
My issue with type hints arises when maintaining them becomes more effort than the benefit they provide. I try to keep a good balance between clarity and simplicity. That’s why s3path doesn’t fully adopt type hints everywhere.
I’ll give you a quick example here, and I’ll leave specific comments in the PR:
Adding a type hint that the init method returns None — I don’t see much value in this.
Specifying that a method returns NoReturn — also seems unnecessary in this context.
Thanks again!
| __slots__ = () | ||
|
|
||
| def __init__(self, *args): | ||
| def __init__(self, *args: Union[str, os.PathLike[str]]) -> None: |
There was a problem hiding this comment.
No need for the return None type hint
| bucket_path = cls(cls.parser.sep, bucket) | ||
| if len(bucket_path.parts) != 2: | ||
| raise ValueError(f'bucket argument contains more then one path element: {bucket_path}') | ||
| key_path = cls(key) | ||
| if key_path.is_absolute(): | ||
| key_path = key_path.relative_to('/') | ||
| return bucket_path / key_path |
There was a problem hiding this comment.
instead of adding the bucket_path,
What do you thing about changing the method hints?
def from_bucket_key(cls, bucket: Union[str, os.PathLike[str]], key: Union[str, os.PathLike[str]]) -> Self:now that I see this class method it make more sense no?
This method can get path object as well
| return uri.replace('file:///', 's3://') | ||
|
|
||
| def _absolute_path_validation(self): | ||
| def _absolute_path_validation(self) -> None: |
There was a problem hiding this comment.
In privet method I'm not sure how much we need type hints
especially when the method don't return just validate and raise exception if needed
|
|
||
| @classmethod | ||
| def cwd(cls): | ||
| def cwd(cls) -> NoReturn: |
There was a problem hiding this comment.
Let's not add this one—I don't see the benefit here.
|
|
||
| @classmethod | ||
| def home(cls): | ||
| def home(cls) -> NoReturn: |
There was a problem hiding this comment.
Let's not add this one—I don't see the benefit here.
| raise | ||
|
|
||
| def is_dir(self) -> bool: | ||
| def is_dir(self, *, follow_symlinks: bool = True) -> bool: |
There was a problem hiding this comment.
default False + validation
And update doc's
| return accessor.is_dir(self) | ||
|
|
||
| def is_file(self) -> bool: | ||
| def is_file(self, *, follow_symlinks: bool = True) -> bool: |
There was a problem hiding this comment.
same comment here about the symlinks
| return False | ||
|
|
||
| def exists(self) -> bool: | ||
| def exists(self, *, follow_symlinks: bool = True) -> bool: |
There was a problem hiding this comment.
same comment here about the symlinks
| return accessor.get_presigned_url(self, expire_in) | ||
|
|
||
| def unlink(self, missing_ok: bool = False): | ||
| def unlink(self, missing_ok: bool = False) -> None: |
| self._absolute_path_validation() | ||
| if not self.key: | ||
| return None | ||
| raise ValueError('Key is required for stat') |
|
@brianhelba You didn't respond to my review so I added most of your changes in a internal PR. |
This adds a number of missing type annotations to the public API surface.
It also fixes the signature of several methods which allow a
follow_symlinksparameter in Python 3.13+. This parameter is allowed inS3Path, but ignored.Unfortunately, these changes are not enough for MyPy to run without errors.