fix(11060): check left_expr in check_generator_or_comprehension - #11254
Closed
hi-ogawa wants to merge 1 commit into
Closed
fix(11060): check left_expr in check_generator_or_comprehension#11254hi-ogawa wants to merge 1 commit into
left_expr in check_generator_or_comprehension#11254hi-ogawa wants to merge 1 commit into
Conversation
This is to detect invalid usage of variables in `AssignmentExpr` hidden inside `left_expr`. Otherwise, such error is shadowed by the inference based on "synthetic generic callable".
Contributor
|
Diff from mypy_primer, showing the effect of this PR on open source code: optuna (https://github.com/optuna/optuna.git)
+ optuna/visualization/_edf.py:138: error: Generator has incompatible item type "FrozenTrial"; expected "Iterable[FrozenTrial]"
+ optuna/visualization/matplotlib/_edf.py:141: error: Generator has incompatible item type "FrozenTrial"; expected "Iterable[FrozenTrial]"
kopf (https://github.com/nolar/kopf.git)
+ kopf/_core/engines/admission.py:441: error: Dict entry 0 has incompatible type "str": "str"; expected "MatchExpression": "str"
+ kopf/_core/engines/admission.py:441: error: Dict entry 1 has incompatible type "str": "str"; expected "MatchExpression": "str"
+ kopf/_core/engines/admission.py:442: error: Dict entry 0 has incompatible type "str": "str"; expected "MatchExpression": "str"
+ kopf/_core/engines/admission.py:442: error: Dict entry 1 has incompatible type "str": "str"; expected "MatchExpression": "str"
+ kopf/_core/engines/admission.py:443: error: Dict entry 0 has incompatible type "str": "str"; expected "MatchExpression": "Sequence[str]"
+ kopf/_core/engines/admission.py:443: error: Dict entry 0 has incompatible type "str": "str"; expected "MatchExpression": "str"
+ kopf/_core/engines/admission.py:443: error: Dict entry 1 has incompatible type "str": "str"; expected "MatchExpression": "Sequence[str]"
+ kopf/_core/engines/admission.py:443: error: Dict entry 1 has incompatible type "str": "str"; expected "MatchExpression": "str"
+ kopf/_core/engines/admission.py:443: error: Dict entry 2 has incompatible type "str": "List[str]"; expected "MatchExpression": "Sequence[str]"
+ kopf/_core/engines/admission.py:443: error: Dict entry 2 has incompatible type "str": "List[str]"; expected "MatchExpression": "str"
mypy (https://github.com/python/mypy.git)
+ mypy/types.py:2139: error: Argument 1 to "accept" of "Type" has incompatible type "TypeStrVisitor"; expected "TypeVisitor[Iterable[str]]" [arg-type]
+ mypy/type_visitor.py:221: error: Argument 1 to "accept" of "Type" has incompatible type "TypeTranslator"; expected "TypeVisitor[List[Type]]" [arg-type]
+ mypy/treetransform.py:477: error: List comprehension has incompatible type List[Expression]; expected List[List[Expression]] [misc]
+ mypy/treetransform.py:488: error: List comprehension has incompatible type List[Expression]; expected List[List[Expression]] [misc]
+ mypy/erasetype.py:146: error: Argument 1 to "accept" of "Type" has incompatible type "LastKnownValueEraser"; expected "TypeVisitor[List[Type]]" [arg-type]
+ mypy/fastparse2.py:1058: note: (Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)
- mypy/fastparse2.py:1058: error: Argument 2 to "visit_Slice" becomes "Any" due to an unfollowed import [no-any-unimported]
- mypy/fastparse2.py:1064: error: Argument 2 to "visit_ExtSlice" becomes "Any" due to an unfollowed import [no-any-unimported]
- mypy/fastparse2.py:1068: error: Argument 2 to "visit_Index" becomes "Any" due to an unfollowed import [no-any-unimported]
ignite (https://github.com/pytorch/ignite)
+ ignite/handlers/time_profilers.py:599: error: Argument 1 to "sum" has incompatible type "List[float]"; expected "Iterable[Iterable[float]]" [arg-type]
xarray (https://github.com/pydata/xarray.git)
+ xarray/core/computation.py:354: error: List comprehension has incompatible type List[Union[Any, List[Any], Mapping[Any, Any]]]; expected List[List[Any]] [misc]
vision (https://github.com/pytorch/vision.git)
+ torchvision/transforms/functional_tensor.py:393: error: Argument 1 to "max" has incompatible type "int"; expected "List[int]" [arg-type]
+ torchvision/transforms/functional_tensor.py:393: error: Argument 2 to "max" has incompatible type "int"; expected "List[int]" [arg-type]
+ torchvision/datasets/omniglot.py:52: error: List comprehension has incompatible type List[str]; expected List[List[str]] [misc]
sockeye (https://github.com/awslabs/sockeye.git)
+ sockeye/data_io.py:93: error: Argument 1 to "max" has incompatible type "int"; expected "List[int]"
+ sockeye/data_io.py:93: error: Argument 2 to "max" has incompatible type "int"; expected "List[int]"
+ sockeye/data_io.py:94: error: Argument 1 to "max" has incompatible type "int"; expected "List[int]"
+ sockeye/data_io.py:94: error: Argument 2 to "max" has incompatible type "int"; expected "List[int]"
+ sockeye/data_io.py:116: error: Argument 1 to "max" has incompatible type "int"; expected "List[int]"
+ sockeye/data_io.py:116: error: Argument 2 to "max" has incompatible type "int"; expected "List[int]"
+ sockeye/beam_search.py:501: error: List comprehension has incompatible type List[int]; expected List[List[List[int]]]
+ sockeye/beam_search.py:501: error: List comprehension has incompatible type List[int]; expected List[List[int]]
PyGithub (https://github.com/PyGithub/PyGithub.git)
+ tests/Connection.py:67: error: Argument 1 to "chain" has incompatible type "*Tuple[Tuple[Type[ReplayingConnection], str], Tuple[str, str]]"; expected "Iterable[Tuple[object, ...]]"
python-chess (https://github.com/niklasf/python-chess.git)
+ chess/__init__.py:3663: error: Argument 1 to "copy" has incompatible type "Move"; expected "List[Move]"
+ chess/gaviota.py:158: error: List comprehension has incompatible type List[int]; expected List[List[int]]
+ chess/gaviota.py:193: error: List item 0 has incompatible type "int"; expected "List[int]"
pandas (https://github.com/pandas-dev/pandas.git)
+ pandas/io/formats/printing.py:491: error: Argument 1 to "max" has incompatible type "int"; expected "List[int]" [arg-type]
+ pandas/io/formats/printing.py:491: error: Argument 2 to "max" has incompatible type "int"; expected "List[int]" [arg-type]
+ pandas/core/strings/accessor.py:3186: error: Argument 2 to "get" of "Mapping" has incompatible type "int"; expected "Union[str, List[Hashable]]" [arg-type]
+ pandas/io/xml.py:285: error: Value expression in dictionary comprehension has incompatible type "Optional[str]"; expected type "str" [misc]
+ pandas/io/parsers/python_parser.py:571: error: List comprehension has incompatible type List[Union[str, int, None]]; expected List[List[Union[str, int, None]]] [misc]
+ pandas/io/formats/style.py:2046: error: Dict entry 0 has incompatible type "str": "str"; expected "Any": "List[CSSDict]" [dict-item]
+ pandas/io/formats/style.py:2047: error: Dict entry 1 has incompatible type "str": "List[Tuple[str, Union[str, int, float]]]"; expected "Any": "List[CSSDict]" [dict-item]
+ pandas/io/formats/style.py:2056: error: Dict entry 0 has incompatible type "str": "str"; expected "Any": "List[CSSDict]" [dict-item]
+ pandas/io/formats/style.py:2057: error: Dict entry 1 has incompatible type "str": "List[Tuple[str, Union[str, int, float]]]"; expected "Any": "List[CSSDict]" [dict-item]
sympy (https://github.com/sympy/sympy.git)
+ sympy/tensor/array/expressions/conv_array_to_matrix.py:755: error: List comprehension has incompatible type List[bool]; expected List[List[int]]
- sympy/parsing/autolev/test-examples/pydy-example-repo/mass_spring_damper.py:10: error: "Callable[[Any, Any, KwArg(Any)], Any]" has no attribute "_t"
+ sympy/parsing/autolev/test-examples/pydy-example-repo/mass_spring_damper.py:10: note: (Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)
websockets (https://github.com/aaugustin/websockets.git)
+ src/websockets/legacy/server.py:775: error: Argument 1 to "create_task" has incompatible type "Coroutine[Any, Any, None]"; expected "Union[Generator[Optional[Future[object]], None, Task[None]], Awaitable[Task[None]]]"
poetry (https://github.com/python-poetry/poetry.git)
- poetry/mixology/failure.py:143: error: Item "PackageNotFoundCause" of "Union[RootCause, NoVersionsCause, DependencyCause, ConflictCause, PythonCause, PlatformCause, PackageNotFoundCause]" has no attribute "other"
- poetry/mixology/failure.py:144: error: Item "RootCause" of "Union[RootCause, NoVersionsCause, DependencyCause, ConflictCause, PythonCause, PlatformCause, PackageNotFoundCause]" has no attribute "conflict"
- poetry/mixology/failure.py:144: error: Item "NoVersionsCause" of "Union[RootCause, NoVersionsCause, DependencyCause, ConflictCause, PythonCause, PlatformCause, PackageNotFoundCause]" has no attribute "conflict"
- poetry/mixology/failure.py:144: note: (Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)
+ poetry/mixology/failure.py:143: note: (Skipping most remaining errors due to unresolved imports or missing stubs; fix these first)
mypy_primer (https://github.com/hauntsaninja/mypy_primer.git)
+ mypy_primer.py:544: error: Argument 1 to "replace" has incompatible type "Project"; expected "Iterator[Project]"
paroxython (https://github.com/laowantong/paroxython.git)
+ paroxython/derived_labels_db.py:221: error: Argument 1 to "map" has incompatible type "Type[str]"; expected "Callable[[Any], List[str]]"
+ paroxython/derived_labels_db.py:222: error: Argument "key" to "max" has incompatible type "Callable[[Sized], int]"; expected "Callable[[Iterable[Any]], SupportsLessThan]"
graphql-core (https://github.com/graphql-python/graphql-core.git)
+ src/graphql/pyutils/frozen_list.py:42: error: Argument 1 to "deepcopy" has incompatible type "T"; expected "Iterable[Any]"
spark (https://github.com/apache/spark.git)
+ python/pyspark/pandas/frame.py:5267: error: Unsupported left operand type for & ("Iterable[Column]") [operator]
|
hi-ogawa
marked this pull request as draft
October 3, 2021 13:30
Contributor
Author
|
Looking at mypy self test and mypy primer's result #11254 (comment), what I did seems to be quite bad, sorry about that... |
Contributor
Author
|
Let me close this since I don't think I can find a proper fix for this. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Fixes #11060 (This PR fixes 2nd example
[z := z + y for y in range(2)]. 1st example is already fixed by #11153)I changed
check_generator_or_comprehensionso that it checksGenerator.left_exprbefore doing "synthetic generic callable" based inference usingcheck_call.It seems that the reason why the error is not reported is that because "synthetic generic callable"s
check_callhides such error due toself.msg.disable_errorsininfer_function_type_arguments. I made a test casetestWalrusUndefinedRecursiveReference4to demonstrate the same issue without using comprehension (which isn't fixed by this PR).So, I attempted to explicitly check
Generator.left_exprindependent fromcheck_calland this seems to fix at least the specific issue with comprehension (and I didn't see any regression in existing tests).I don't believe this change is a legitimate fix for the issue, but I thought this is something worth sharing, so I made a PR here. I hope you don't mind and I would appreciate any feedback about this approach.
Thanks a lot!
Test Plan
I added
testWalrusUndefinedRecursiveReference1, 2, 3, 4. The errors in1and3were already detected.2is the one this PR fixed.4is not handled correctly and I think it demonstrates some issue about function call inference withTypeVar.