-
Notifications
You must be signed in to change notification settings - Fork 671
feat(api): Make RESTManager generic on RESTObject class #3083
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
This is a lot more verbose than #3080 but I think it is a better solution with less hacks and I did not make a RESTObjectList a generic or add overloads to list methods here to keep this PR more focused on RESTManager but I want to do a follow-up PR in the future. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3083 +/- ##
==========================================
- Coverage 97.28% 97.25% -0.04%
==========================================
Files 97 97
Lines 6006 5932 -74
==========================================
- Hits 5843 5769 -74
Misses 163 163
Flags with carried forward coverage won't be shown. Click here to find out more.
|
I also have a 3rd implementation there I made RESTManager a Protocol as described in this comment It allows mixins to be "mixins" instead of subclasses using However, it quickly ran in to several issues:
|
There is also an idea about SidekiqManager. Set its |
c7b1d3d
to
34e73f4
Compare
76bc2b2
to
8900a0f
Compare
8900a0f
to
2bc8bb5
Compare
Thanks a lot for all the work here @igorp-collabora. Technically your 3rd solution would be really clean IMO and would be nice to have proper mixins and IIRC that is also how I imagined protocols last I played around with this. But, I guess they are not real mixins from mypy's perspective anyway whenever we rely on specific instance attributes 😁 The PRs are actually quite reasonable to review but the main chunk is the removal of the manual get overrides, which makes everything a huge change. So I'd be open to reviewing that separately if that works for you (1 PR to add native typing, which should already solve it for other methods, and another for removing the custom overrides and test). Let me know if that works. It would be slightly easier to focus on the details then 🙇 I like the idea of defining a path on the python-gitlab/gitlab/v4/objects/topics.py Line 60 in 36d9b24
|
I can do that but I want to check if keeping the
I will add this. |
@igorp-collabora separate commits but the same PR would also work for me if that's ok! 🙇 |
2bc8bb5
to
774b3f9
Compare
774b3f9
to
4e3fe18
Compare
Things should be a bit calmer now after the backlog merge and release(s) so hopefully less conflict/rebasing here 😀 |
4e3fe18
to
f3cddd2
Compare
I rebased the PR and updated all the new RESTManager classes added. I will split the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I managed to get through the diff and left some ramblings (some are non-blocking, just general notes).
In general I like that this will give users accurate typing for most resources.
It might feel a bit odd that our mixins are no longer mixins at all (not just in the typing context), but seems to be more maintainable than the protocol approach for now and the user value is more important 👍 We can revisit the protocol approach at some point if we want the code closer to the user (gitlab.v4.objects.*
) to be less type-verbose but IMO this is OK.
Not sure if @JohnVillalovos would also like to give this another look as well 🙇
4e33522
to
b9a1325
Compare
@nejch I think this PR is in good enough form. The only possible change I might add is splitting the I think there is also room to add a new unit test that verifies that If you think this PR is good enough. I can start working on splitting the |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nejch I think this PR is in good enough form. The only possible change I might add is splitting the
RESTManager
in to 2 classes. (the parent one with just_path
and one that adds_obj_cls
)I think there is also room to add a new unit test that verifies that
_obj_cls
and_path
are properly set for subclasses. Since they are no longer defined using@abstractmethod @property
the type checkers will not verify that they properly defined.
@igorp-collabora agree, I think this is almost good to go! Now that I can focus just on a few files I just found a few more nits that I forgot earlier. I think the base manager could work well if it helps with the code using it. Let me know if that's ok.
I'm happy to see typing and linting here helps get rid of our custom checks so if possible I'd prefer to avoid more of them - maybe we can have a follow-up for that though.
1de5b6b
to
eedbf9d
Compare
@nejch I'm not sure if I will have time to review this in the near future. I trust your reviews though 👍🤓 |
@nejch After this gets merged, I will propose a PR to update the black config so that it will do the equivalent of Basically add to
|
I made a separate PR to remove the Once it is merged I will rebase this PR. It should shrink by quite a bit after that. |
eedbf9d
to
a6e9333
Compare
Rebase done. The line diff is now |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This LGTM, I'll just wait with merging until later today so I can do a checkout on some of our projects that make heavy use of the library. Also it'll be daytime in Oregon by then in case @JohnVillalovos has any objections 😁
This should have no impact at runtime but if mypy starts complaining a lot, it might be good if we add a breaking change so people can migrate more easily when they get their dependency/renovate bot bumps.
Thanks a lot for your patience @igorp-collabora and for all the effort put in here!
I think there shouldn't be any difference with the Get methods because of redefined get methods had the same effective type hints, however, other mixins might make certain code no longer pass type checking. Biggest changes that might impact type checking:
|
No objections from me. Sorry I haven't had the time to review this yet. I trust your decision @nejch |
Currently mixins like ListMixin are type hinted to return base RESTObject instead of a specific class like `MergeRequest`. The GetMixin and GetWithoutIdMixin solve this problem by defining a new `get` method for every defined class. However, this creates a lot of duplicated code. Make RESTManager use `typing.Generic` as its base class and use and assign the declared TypeVar to the `_obj_cls` attribute as a type of the passed class. Make both `_obj_cls` and `_path` attributes an abstract properties so that type checkers can check that those attributes were properly defined in subclasses. Mypy will only check then the class is instantiated which makes non-final subclasses possible. Unfortunately pylint will check the declarations not instantiations so add `# pylint: disable=abstract-method` comments to all non-final subclasses like ListMixin. Make `_path` attribute always be `str` instead of sometimes `None`. This eliminates unnecessary type checks. Change all mixins like ListMixin or GetMixin to a subclass. This makes the attribute declarations much cleaner as for example `_list_filters` is now the only attribute defined by ListMixin. Change SidekiqManager to not inherit from RESTManager and only copy its `__init__` method. This is because SidekiqManager never was a real manager and does not define `_path` or `_obj_cls`. Delete `tests/unit/meta/test_ensure_type_hints.py` file as the `get` method is no required to be defined for every class. Signed-off-by: Igor Ponomarev <[email protected]>
a6e9333
to
40eb4d0
Compare
Once this gets merged I plan on adding more typing changes:
|
Thanks @igorp-collabora. I checked out python-gitlab from your branch in a few of our projects and generally it all looks great but indeed the list methods are now failing checks where we previously had workarounds for the union types. I think just for clarity I'd make that a breaking change, but we can do this in a follow-up since the I'll go ahead and merge this for now! |
Currently mixins like ListMixin are type hinted to return base RESTObject instead of a specific class like
MergeRequest
.The GetMixin and GetWithoutIdMixin solve this problem by defining a new
get
method for every defined class. However, this creates a lot of duplicated code.Make RESTManager use
typing.Generic
as its base class and use and assign the declared TypeVar to the_obj_cls
attribute as a type of the passed class.Make both
_obj_cls
and_path
attributes an abstract properties so that type checkers can check that those attributes were properly defined in subclasses. Mypy will only check then the class is instantiated which makes non-final subclasses possible. Unfortunately pylint will check the declarations not instantiations so add# pylint: disable=abstract-method
comments to all non-final subclasses like ListMixin.Make
_path
attribute always bestr
instead of sometimesNone
. This eliminates unnecessary type checks.Change all mixins like ListMixin or GetMixin to a subclass. This makes the attribute declarations much cleaner as for example
_list_filters
is now the only attribute defined by ListMixin.Change SidekiqManager to not inherit from RESTManager and only copy its
__init__
method. This is because SidekiqManager never was a real manager and does not define_path
or_obj_cls
.Delete
tests/unit/meta/test_ensure_type_hints.py
file as theget
method is no required to be defined for every class.Closes #2228
Closes #3080