Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Fixes mypy crash on protocol with contravariant var #11135

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JukkaL merged 3 commits into from
Sep 21, 2021
Merged

Fixes mypy crash on protocol with contravariant var #11135

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2b8617b
Fixes mypy crash on protocol with contravariant var
sobolevn Sep 18, 2021
957922b
Fixes how `infer_constraints_from_protocol_members` works.
sobolevn Sep 20, 2021
e5f80ca
Adds positive case
sobolevn Sep 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions mypy/constraints.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -388,7 +388,7 @@ def visit_instance(self, template: Instance) -> List[Constraint]:
if (template.type.is_protocol and self.direction == SUPERTYPE_OF and
# We avoid infinite recursion for structural subtypes by checking
# whether this type already appeared in the inference chain.
# This is a conservative way break the inference cycles.
# This is a conservative way to break the inference cycles.
# It never produces any "false" constraints but gives up soon
# on purely structural inference cycles, see #3829.
# Note that we use is_protocol_implementation instead of is_subtype
Expand All @@ -398,8 +398,8 @@ def visit_instance(self, template: Instance) -> List[Constraint]:
for t in template.type.inferring) and
mypy.subtypes.is_protocol_implementation(instance, erased)):
template.type.inferring.append(template)
self.infer_constraints_from_protocol_members(res, instance, template,
original_actual, template)
res.extend(self.infer_constraints_from_protocol_members(
instance, template, original_actual, template))
template.type.inferring.pop()
return res
elif (instance.type.is_protocol and self.direction == SUBTYPE_OF and
Expand All @@ -408,8 +408,8 @@ def visit_instance(self, template: Instance) -> List[Constraint]:
for i in instance.type.inferring) and
mypy.subtypes.is_protocol_implementation(erased, instance)):
instance.type.inferring.append(instance)
self.infer_constraints_from_protocol_members(res, instance, template,
template, instance)
res.extend(self.infer_constraints_from_protocol_members(
instance, template, template, instance))
instance.type.inferring.pop()
return res
if isinstance(actual, AnyType):
Expand All @@ -432,26 +432,30 @@ def visit_instance(self, template: Instance) -> List[Constraint]:
else:
return []

def infer_constraints_from_protocol_members(self, res: List[Constraint],
def infer_constraints_from_protocol_members(self,
instance: Instance, template: Instance,
subtype: Type, protocol: Instance) -> None:
subtype: Type, protocol: Instance,
) -> List[Constraint]:
"""Infer constraints for situations where either 'template' or 'instance' is a protocol.

The 'protocol' is the one of two that is an instance of protocol type, 'subtype'
is the type used to bind self during inference. Currently, we just infer constrains for
every protocol member type (both ways for settable members).
"""
res = []
for member in protocol.type.protocol_members:
inst = mypy.subtypes.find_member(member, instance, subtype)
temp = mypy.subtypes.find_member(member, template, subtype)
assert inst is not None and temp is not None
if inst is None or temp is None:

@JelleZijlstra JelleZijlstra Sep 18, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with this code. Do you know why it is safe to ignore it?

@sobolevn sobolevn Sep 18, 2021 •
edited
Loading

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a pretty old piece from initial protocols commit c55e48b

I was not sure if this is safe to ignore or not. All code / tests I've tried showed that nothing has changed.

But, from personal experience finding bugs in constraints solver is pretty hard.

We can ask @ilevkivskyi if he knows some examples / cases where this can backfire.

@ilevkivskyi ilevkivskyi Sep 19, 2021

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a basic consistency check. One should never get here for something that can never be a protocol implementation. So I don't think this fixes the actual issue. Couple other comments:

  • Variance in the place where it is used in the original repro should be totally irrelevant. Variance is only relevant for class definition type variables. It should be irrelevant in generic function/method type variables (including self-types). Variance is a property of a class, not a property of a type variable. If the repro depends on variance we have a much bigger problem.
  • I think the actual culprit is overload on self-type. This is a relatively new feature (much newer than protocols), and it requires some special handling in bind_self() (and/or somewhere around). I bet find_member() doesn't handle this logic correctly. So you should either fix that, or better refactor find_member() somehow to use the same logic.

return [] # See #11020
# The above is safe since at this point we know that 'instance' is a subtype
# of (erased) 'template', therefore it defines all protocol members
res.extend(infer_constraints(temp, inst, self.direction))
if (mypy.subtypes.IS_SETTABLE in
mypy.subtypes.get_member_flags(member, protocol.type)):
# Settable members are invariant, add opposite constraints
res.extend(infer_constraints(temp, inst, neg_op(self.direction)))
return res

def visit_callable_type(self, template: CallableType) -> List[Constraint]:
if isinstance(self.actual, CallableType):
Expand Down
60 changes: 59 additions & 1 deletion test-data/unit/check-protocols.test
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -486,6 +486,64 @@ class P2(Protocol[T_co]): # E: Covariant type variable "T_co" used in protocol w
lst: List[T_co]
[builtins fixtures/list.pyi]


[case testProtocolConstraintsUnsolvableWithSelfAnnotation1]
# https://github.com/python/mypy/issues/11020
from typing import overload, Protocol, TypeVar

I = TypeVar('I', covariant=True)
V_contra = TypeVar('V_contra', contravariant=True)

class C(Protocol[I]):
def __abs__(self: 'C[V_contra]') -> 'C[V_contra]':
...

@overload
def f(self: 'C', q: int) -> int:
...
@overload
def f(self: 'C[float]', q: float) -> 'C[float]':
...
[builtins fixtures/bool.pyi]


[case testProtocolConstraintsUnsolvableWithSelfAnnotation2]
# https://github.com/python/mypy/issues/11020
from typing import Protocol, TypeVar

I = TypeVar('I', covariant=True)
V = TypeVar('V')

class C(Protocol[I]):
def g(self: 'C[V]') -> 'C[V]':
...

class D:
pass

x: C = D() # E: Incompatible types in assignment (expression has type "D", variable has type "C[Any]")
[builtins fixtures/bool.pyi]


[case testProtocolConstraintsUnsolvableWithSelfAnnotation3]
# https://github.com/python/mypy/issues/11020
from typing import Protocol, TypeVar

I = TypeVar('I', covariant=True)
V = TypeVar('V')

class C(Protocol[I]):
def g(self: 'C[V]') -> 'C[V]':
...

class D:
def g(self) -> D:
...

x: C = D()
[builtins fixtures/bool.pyi]


[case testProtocolVarianceWithUnusedVariable]
from typing import Protocol, TypeVar
T = TypeVar('T')
Expand Down Expand Up @@ -2124,7 +2182,7 @@ class B(Protocol):
def execute(self, stmt: Any, *args: Any, **kwargs: Any) -> None: ...
def cool(self) -> None: ...

def func1(arg: A) -> None: ...
def func1(arg: A) -> None: ...
def func2(arg: Optional[A]) -> None: ...

x: B
Expand Down