From 97167d02ddd0d11a851107dabc93c9d5080be26d Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 26 Nov 2021 10:36:03 +0000 Subject: [PATCH 1/5] Improve documentation for annotating generator functions The mypy documentation is currently ambivalent on whether ``Iterator`` or ``Iterable`` is the better return-type annotation for a simple generator function. The documentation should endorse using ``Iterator`` instead of ``Iterable`, in my opinion, as it is a more precise return type, and no less concise. --- docs/source/kinds_of_types.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index b1b6648b0c1cc..f72b1b80951a1 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -721,8 +721,8 @@ For more details, see :ref:`type-variable-value-restriction`. Generators ********** -A basic generator that only yields values can be annotated as having a return -type of either :py:class:`Iterator[YieldType] ` or :py:class:`Iterable[YieldType] `. For example: +A basic generator that only yields values can be succinctly annotated as having a return +type of :py:class:`Iterator[YieldType] `. For example: .. code-block:: python @@ -755,7 +755,7 @@ annotated the first example as the following: for i in range(n): yield i * i -This is slightly different from using ``Iterable[int]`` or ``Iterator[int]``, +This is slightly different from using ``Iterator[int]`` or ``Iterable[int]``, since generators have :py:meth:`~generator.close`, :py:meth:`~generator.send`, and :py:meth:`~generator.throw` methods that -generic iterables don't. If you will call these methods on the returned -generator, use the :py:class:`~typing.Generator` type instead of :py:class:`~typing.Iterable` or :py:class:`~typing.Iterator`. +generic iterators and iterables don't. If you plan to call these methods on the returned +generator, use the :py:class:`~typing.Generator` type instead of :py:class:`~typing.Iterator`. From c5a7c31522fd8bb1f26bcfd635d54c4b24ecd8ec Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 26 Nov 2021 14:12:45 +0000 Subject: [PATCH 2/5] Address review --- docs/source/kinds_of_types.rst | 21 +++++++++++++++++---- 1 file changed, 17 insertions(+), 4 deletions(-) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index f72b1b80951a1..c001aa53ac336 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -722,17 +722,30 @@ Generators ********** A basic generator that only yields values can be succinctly annotated as having a return -type of :py:class:`Iterator[YieldType] `. For example: +type of either :py:class:`Iterator[YieldType] ` or :py:class:`Iterable[YieldType] `. For example: .. code-block:: python def squares(n: int) -> Iterator[int]: for i in range(n): yield i * i + +In general, :py:class:`Iterator[YieldType] ` should be preferred over +:py:class:`Iterable[YieldType] ` for the return type of a generator function, +as it is more precise regarding the type of the returned object. All iterator objects have a +:py:meth:`~iterator.__next__` method, so using ``Iterator`` as the return type alerts mypy to +the fact that :py:func:`next` can be called on the object returned from the generator function. +Using ``Iterable`` runs the risk of mypy raising false-positive errors if a user attempts to call +`next()` on the returned object. + +However, in some cases, it may be considered an implementation detail that ``next()`` can be +called on the object returned from the generator function. In these cases, it may be preferable +to use ``Iterable`` instead of ``Iterator``. If you want your generator to accept values via the :py:meth:`~generator.send` method or return -a value, you should use the -:py:class:`Generator[YieldType, SendType, ReturnType] ` generic type instead. For example: +a value, meanwhile, you should use the +:py:class:`Generator[YieldType, SendType, ReturnType] ` generic type instead of +either ``Iterator`` or ``Iterable``. For example: .. code-block:: python @@ -758,4 +771,4 @@ annotated the first example as the following: This is slightly different from using ``Iterator[int]`` or ``Iterable[int]``, since generators have :py:meth:`~generator.close`, :py:meth:`~generator.send`, and :py:meth:`~generator.throw` methods that generic iterators and iterables don't. If you plan to call these methods on the returned -generator, use the :py:class:`~typing.Generator` type instead of :py:class:`~typing.Iterator`. +generator, use the :py:class:`~typing.Generator` type instead of :py:class:`~typing.Iterator` or :py:class:`~typing.Iterable`. From 827ae8b6e9b3e044fd8bd6df6b33ad7b13afb93c Mon Sep 17 00:00:00 2001 From: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Date: Fri, 26 Nov 2021 15:03:48 -0800 Subject: [PATCH 3/5] Update docs/source/kinds_of_types.rst --- docs/source/kinds_of_types.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index c001aa53ac336..6bc5a9453ec43 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -743,7 +743,7 @@ called on the object returned from the generator function. In these cases, it ma to use ``Iterable`` instead of ``Iterator``. If you want your generator to accept values via the :py:meth:`~generator.send` method or return -a value, meanwhile, you should use the +a value, you should use the :py:class:`Generator[YieldType, SendType, ReturnType] ` generic type instead of either ``Iterator`` or ``Iterable``. For example: From d1b75a02824bcfece5f3ff8fd95dfe0618edfefd Mon Sep 17 00:00:00 2001 From: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Date: Fri, 26 Nov 2021 15:34:53 -0800 Subject: [PATCH 4/5] Update docs/source/kinds_of_types.rst --- docs/source/kinds_of_types.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index 6bc5a9453ec43..42f4cca6b8422 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -743,7 +743,7 @@ called on the object returned from the generator function. In these cases, it ma to use ``Iterable`` instead of ``Iterator``. If you want your generator to accept values via the :py:meth:`~generator.send` method or return -a value, you should use the +a value, however, you should use the :py:class:`Generator[YieldType, SendType, ReturnType] ` generic type instead of either ``Iterator`` or ``Iterable``. For example: From 9c84d21bef0773e68624dd49ff7b9a459dedfe85 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Fri, 26 Nov 2021 23:56:58 +0000 Subject: [PATCH 5/5] Make more succinct, addressing review --- docs/source/kinds_of_types.rst | 22 ++++++++++------------ 1 file changed, 10 insertions(+), 12 deletions(-) diff --git a/docs/source/kinds_of_types.rst b/docs/source/kinds_of_types.rst index 42f4cca6b8422..7cdedf6718670 100644 --- a/docs/source/kinds_of_types.rst +++ b/docs/source/kinds_of_types.rst @@ -730,20 +730,18 @@ type of either :py:class:`Iterator[YieldType] ` or :py:class:`I for i in range(n): yield i * i -In general, :py:class:`Iterator[YieldType] ` should be preferred over -:py:class:`Iterable[YieldType] ` for the return type of a generator function, -as it is more precise regarding the type of the returned object. All iterator objects have a -:py:meth:`~iterator.__next__` method, so using ``Iterator`` as the return type alerts mypy to -the fact that :py:func:`next` can be called on the object returned from the generator function. -Using ``Iterable`` runs the risk of mypy raising false-positive errors if a user attempts to call -`next()` on the returned object. - -However, in some cases, it may be considered an implementation detail that ``next()`` can be -called on the object returned from the generator function. In these cases, it may be preferable -to use ``Iterable`` instead of ``Iterator``. +A good rule of thumb is to annotate functions with the most specific return +type possible. However, you should also take care to avoid leaking implementation +details into a function's public API. In keeping with these two principles, prefer +:py:class:`Iterator[YieldType] ` over +:py:class:`Iterable[YieldType] ` as the return-type annotation for a +generator function, as it lets mypy know that users are able to call :py:func:`next` on +the object returned by the function. Nonetheless, bear in mind that ``Iterable`` may +sometimes be the better option, if you consider it an implementation detail that +``next()`` can be called on the object returned by your function. If you want your generator to accept values via the :py:meth:`~generator.send` method or return -a value, however, you should use the +a value, on the other hand, you should use the :py:class:`Generator[YieldType, SendType, ReturnType] ` generic type instead of either ``Iterator`` or ``Iterable``. For example: