Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add a covariant Mapping-like type #5

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
harahu opened this issue May 19, 2022 · 13 comments
Closed

Add a covariant Mapping-like type #5

harahu opened this issue May 19, 2022 · 13 comments

Comments

@harahu
Copy link

harahu commented May 19, 2022

What

I want a new protocol added to typing or typing_extensions, expressing the aspects of Mapping that are covariant with regards to the key type. That is, the Mapping type, but without the __getitem__ and get methods.

Something along the lines of:

class CovariantMappingLike(Protocol[KT_co, VT_co]):
    # A better name for this protocol might be needed
    def __eq__(self, other: object) -> bool: ...
    def __ne__(self, other: object) -> bool: ...
    def keys(self) -> KeysView[KT_co]: ...
    def items(self) -> ItemsView[KT_co, VT_co]: ...
    def values(self) -> ValuesView[VT_co]: ...

Why

Mappings are often, in practice, treated in ways that make them covariant with regards to key. A specific example is laid out in a comment of mine found here: python/typing#445 (comment)

The Mapping type, however, cannot be covariant w. regards to key, as it also implements methods that don't allow for it (__getitem__ and get) to be covariant.

This leaves me unable to express, in a simple and idiomatic way, that I intend to use a Mapping in a way that respects covariance.

Being able to write:

def foo(mapping: CovariantMappingLike[Hashable, Hashable]) -> int:
    ...

would allow me to express what I'm after. It would allow me to express: "Give me a mapping, but I promise to not be looking up keys on it."

This, again, makes it possible to have narrower key types for my Mapping objects, as I can pass a narrowly typed Mapping to foo, without casting:

my_map: Dict[Literal["foo", "bar", "baz"], str] = {}

# I want this to be fine
foo(my_mapping)

I was prompted by @srittau to create this issue in python/typing#445 (comment)

@harahu harahu changed the title Add a covariant mapping-like type Add a covariant Mapping-like type May 19, 2022
@srittau
Copy link
Collaborator

srittau commented May 19, 2022

As I wrote in the original issue, I think this would be quite useful in addition to our tight protocols. Maybe we could have a look at the use of Mapping and the tighter protocols in typeshed to determine, which methods are best suited for such a protocol and add it to typing_extensions first to gain some experience.

@harahu
Copy link
Author

harahu commented May 19, 2022

The proposal I made, was intended to be "as broad as possible, but without breaking covariance". In light of that, I should probably inherit from Container as well. Unsure if there are more methods one could add, while maintaining covariance, than the ones I listed.

The reason I want a type that is "as broad as possible", has to do with usability. Tight protocols can, unfortunately, be cumbersome, in some situations, as long as we lack intersection types.

A concrete (but perhaps silly) example of a function in need of annotation:

def do_more_than_one_thing_with_a_mapping(mapping):
   # The exact methods called on the mapping are unimportant. 
   # The salient detail is that there's more than one method being called.
   return any(mapping.keys()), all(mapping.values())

And three ways of annotating it:

  1. Not currently possible, but what I hope Python 3.13 will look like:
def silly_fn(mapping: SupportsKeys & SupportsItems) -> tuple[bool, bool]:
    ...
  1. The most direct equivalent possible today:
class SupportsKeysAndItems(SupportsKeys[KT_co], SupportsItems[VT_co], Protocol[KT_co, VT_co]):
    # A full extra class definition, just to create an intersection type(!)
    ...

def silly_fn(mapping: SupportsKeysAndItems) -> tuple[bool, bool]:
    ...
  1. Using a type that is broader in scope than it needs to be, but avoiding having to manually create the correct intersection type:
def silly_fn(mapping: CovariantMappingLike) -> tuple[bool, bool]:
    ...

My preference looks like: 1 > 3 > 2, so until we can have 1, I'd prefer having the option of 3 available when calling more than one method on the mapping.

Not trying to say that narrow protocols are bad. They can be really useful when annotating simple functions. So I want them around. It's just that the syntax for composing them is so bad that I'd like someone else to do it for me, neatly hidden away in some library.

@JelleZijlstra JelleZijlstra transferred this issue from python/typing May 19, 2022
@gvanrossum
Copy link
Member

I feel this is too rarely needed to add here. You can just define a protocol in your own code.

@harahu
Copy link
Author

harahu commented Jun 4, 2022

I feel this is too rarely needed to add here. You can just define a protocol in your own code.

The need to interact co-variantly with mapping keys seems pretty common to me. See:

This type can be used for every function that interacts with a mapping without looking up values, i.e. covering all "safe" use-cases of what's suggested in the issues mentioned above. That's a pretty good compromise solution IMO.

The Mapping stubs in typeshed can also be defined based on this type, making for a neat simplification, and also isolates the part of the Mapping API that leads to invariance, which, I think, is a good idea.

@gvanrossum
Copy link
Member

If we have to have it, maybe CovariantMapping?

@harahu
Copy link
Author

harahu commented Jun 4, 2022

If we have to have it, maybe CovariantMapping?

One objection to that name is that it can sound a bit like a subtype of Mapping. But in this case it's a supertype. Not sure how to convey that elegantly.

@gvanrossum
Copy link
Member

I don't think you can get the perfect name, so I recommend not overthinking it.

@hmc-cs-mdrissi
Copy link

hmc-cs-mdrissi commented Jun 5, 2022

My main question is when would you choose covariant mapping vs mapping with a typevar for key? What's difference between,

def foo(mapping: CovariantMapping[Hashable, Hashable]):
  ...

vs

HashableT = TypeVar('HashableT', bound=Hashable)

def foo(mapping: Mapping[HashableT, Hashable]):
  ...

I occasionally use latter as a way to make an invariant type argument behave as if it were covariant.

@harahu
Copy link
Author

harahu commented Jun 5, 2022

I occasionally use latter as a way to make an invariant type argument behave as if it were covariant.

I have, and am using the same pattern. But it is both more verbose, and harder to understand for the reader.

Given that most documentation for TypeVars showcase their purpose as illustrating dependencies between types in more than one location, I think one need to expect the reader to have a pretty strong understanding of the concept before they'll be able to effortlessly read you second example.

That is, for the inexperienced reader, seeing: Mapping[HashableT, Hashable] without seeing HashableT appear a second time in the function definition, has the potential of creating confusion. This shouldn't be an issue with CovariantMapping[Hashable, Hashable]

@carljm
Copy link
Member

carljm commented Mar 15, 2023

I agree that the need for a mapping type that is covariant in its key does come up regularly, but a mapping protocol that doesn't implement __getitem__ at all seems too limited; is it really that common to use a mapping and never index into it? The common use cases would seem to be limited to when you are just iterating over the keys and values in the mapping, or comparing it to another mapping. Indexing seems pretty fundamental to what it means to be a mapping.

I think a more useful "covariant in keys" mapping type would be one that does provide __getitem__ but has it accept object instead of the key type, as originally proposed as option 2 in python/typing#445. Without typechecker special casing, this means you effectively give up type-checking of indices (so the type checker won't catch m[3] as an error if m: CovariantMapping[str, str] -- IMO this is not a big deal, as this is arguably not even a type error, just a key error), but it allows the mapping to be covariant in the key type and still permit indexing.

I think it's too late to make this change for Mapping, but if we do introduce a new CovariantMapping, I think it would be more useful this way.

@JelleZijlstra
Copy link
Member

Closing this following the policy to only include things that are in CPython or in a PEP.

@JelleZijlstra JelleZijlstra closed this as not planned Won't fix, can't repro, duplicate, stale Jun 2, 2023
@harahu
Copy link
Author

harahu commented Jun 2, 2023

Closing this following the policy to only include things that are in CPython or in a PEP.

Is there an alternative location where these non-PEP, but generally usable types could be shared?

@JelleZijlstra
Copy link
Member

We'll have to make a new package; I think @hauntsaninja is planning to do that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants