When writing plugins for code which uses Annotated[Type, an_obj, ..., another_obj] types dynamically, it would be nice to be able to access these annotations in order to either (a) perform type-checking on them, and/or (b) use those dynamic annotations to correctly infer the existence or types of other objects. I'm a total newbie to the mypy internals, but I think this is not currently possible. (See also issue #10872, and PRs #9625, #10777.) It would be fantastic if mypy could support the proper treatment of later parameters of Annotated somehow.
If I understand the current situation correctly, by the time the plugin gets a hold of the type annotation of something which was Annotated, even the unanalyzed_type, mypy will have run the TypeConverter on all of the parameters of the Annotated[...] annotation. This unfortunately erases most of the information in them, rendering them useless to the plugin. Note that PEP 593 states
Annotated is parameterized with a type and an arbitrary list of Python values that represent the annotations.
so it is in principle wrong to treat all of the parameters of Annotated[...] as types anyway.
However, I presume the parser cannot easily tell whether a given Subscript node in the AST is an Annotated[...] field in the relevant part of fastparse.py where this type conversion happens. That is, it cannot really distinguish between dict[str, "Something"] and Annotated[str, "Something"] in order to treat these cases differently (i.e. the first refers to a type Something which happens to be in a string, and the second simply contains a string "Something").
I'm not familiar enough with mypy internals to know what the best solution is. The fundamental tension that mypy wants to do type conversion directly on the AST, but it's not until semantic analysis starts that it knows which nodes are Annotated. I can see that during semantic analysis, expr_to_unanalyzed_type is used. Given that this is possible, it seems to me like the most natural thing is if the AST parsing stage does not do type conversion on Subscript parameters, but instead converts them to Expressions which are later converted to types if appropriate. Presumably this comes at some performance cost, though, in the more common cases of parameters being types (dict[str, int] or list[str] etc.).
Any thoughts would be fantastic.
When writing plugins for code which uses
Annotated[Type, an_obj, ..., another_obj]types dynamically, it would be nice to be able to access these annotations in order to either (a) perform type-checking on them, and/or (b) use those dynamic annotations to correctly infer the existence or types of other objects. I'm a total newbie to the mypy internals, but I think this is not currently possible. (See also issue #10872, and PRs #9625, #10777.) It would be fantastic if mypy could support the proper treatment of later parameters ofAnnotatedsomehow.If I understand the current situation correctly, by the time the plugin gets a hold of the type annotation of something which was
Annotated, even theunanalyzed_type, mypy will have run theTypeConverteron all of the parameters of theAnnotated[...]annotation. This unfortunately erases most of the information in them, rendering them useless to the plugin. Note that PEP 593 statesso it is in principle wrong to treat all of the parameters of
Annotated[...]as types anyway.However, I presume the parser cannot easily tell whether a given
Subscriptnode in the AST is anAnnotated[...]field in the relevant part of fastparse.py where this type conversion happens. That is, it cannot really distinguish betweendict[str, "Something"]andAnnotated[str, "Something"]in order to treat these cases differently (i.e. the first refers to a typeSomethingwhich happens to be in a string, and the second simply contains a string"Something").I'm not familiar enough with mypy internals to know what the best solution is. The fundamental tension that mypy wants to do type conversion directly on the AST, but it's not until semantic analysis starts that it knows which nodes are Annotated. I can see that during semantic analysis,
expr_to_unanalyzed_typeis used. Given that this is possible, it seems to me like the most natural thing is if the AST parsing stage does not do type conversion onSubscriptparameters, but instead converts them toExpressions which are later converted to types if appropriate. Presumably this comes at some performance cost, though, in the more common cases of parameters being types (dict[str, int]orlist[str]etc.).Any thoughts would be fantastic.