Thanks to visit codestin.com
Credit goes to github.com

Skip to content

surprising new behavior in inspect/annotationlib get_annotations as of 3.14.0b1, not in 3.14.0a7. intended change? #133684

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
zzzeek opened this issue May 8, 2025 · 19 comments
Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir topic-typing type-bug An unexpected behavior, bug, or error

Comments

@zzzeek
Copy link

zzzeek commented May 8, 2025

Bug report

Bug description:

3.14.0b1 seems to be interpreting the annotations of a metaclass into a new format that is also interfering with the correct annotations of subclasses of that class. The subclass reports the annotations of the parent class, if it does not itself have any annotations. this only occurs if superclass is from a custom metaclass. if the subclass does have annotations, then the superclass annotations are not reported. this behavior is inconsistent with previous python versions including 3.14.0a7 and is also inconsistent with itself. see script below

from __future__ import annotations


import inspect

# a metaclass with an annotation
class MyMetaClass(type):
    metadata: dict[str, str]


# dynamic class based on metaclass
MyNewClass = MyMetaClass("MyNewClass", (object,), {"metadata": {}})

# this seems new, but maybe intentional.   annotations are shown for the
# dynamic class that come from the metaclass.  OK
# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {'metadata': 'dict[str, str]'}
print(inspect.get_annotations(MyNewClass))


# declared class based on metaclass.   Add another anno to it.
class MyOtherNewClass(metaclass=MyMetaClass):
    metadata: dict[str, str] = {}
    someint: int

# both pythons look good and this agrees with older pythons too.
# 3.13.3: {'metadata': 'dict[str, str]', 'someint': 'int'}
# 3.14.0a7: {'metadata': 'dict[str, str]', 'someint': 'int'}
# 3.14.0b1: {'metadata': 'dict[str, str]', 'someint': 'int'}
print(inspect.get_annotations(MyOtherNewClass))

# a control, a normal class without special metaclass.
class YetAnotherClass:
    metadata: dict[str, str] = {}
    someint: int



# here's where it goes wrong.   If we make new classes from these
# bases that have *no* annotations, the *superclass annotations leak into the subclass*

class MySubClass(MyNewClass):
    pass


# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {'metadata': 'dict[str, str]', 'someint': 'int'}  # this has to be wrong
print(inspect.get_annotations(MySubClass))

# if we add *any* annotations to MySubClass, the above annos *disappear*
class MySubClass(MyNewClass):
    foobar: float

# these look all correct.  but 3.14.0b1's seems to be inconsistent with itself
# 3.13.3: {'foobar': 'float'}
# 3.14.0a7: {'foobar': 'float'}
# 3.14.0b1: {'foobar': 'float'}  # wait what?  What happened to metadata/someint from above?
print(inspect.get_annotations(MySubClass))



class MyOtherSubClass(MyOtherNewClass):
    pass

# similar behaviors for declared class
# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {'metadata': 'dict[str, str]', 'someint': 'int'}
print(inspect.get_annotations(MyOtherSubClass))


# behavior does not seem to occur at all without a metaclass.
# YetAnotherClass has annotations but we do not see them from subclasses
# of that class
class MyYetAnotherSubClass(YetAnotherClass):
    pass

# 3.13.3: {}
# 3.14.0a7: {}
# 3.14.0b1: {}
print(inspect.get_annotations(MyYetAnotherSubClass))

outputs:

$ python test4.py    # python 3.13.3
{}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{}
{'foobar': 'float'}
{}
{}

$ ~/.venv314a7/bin/python test4.py   # python 3.14.0a7
{}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{}
{'foobar': 'float'}
{}
{}

$ ~/.venv314/bin/python test4.py   # python 3.14.0b1
{'metadata': 'dict[str, str]'}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{'metadata': 'dict[str, str]'}
{'foobar': 'float'}
{'metadata': 'dict[str, str]', 'someint': 'int'}
{}


CPython versions tested on:

3.14

Operating systems tested on:

No response

Linked PRs

@zzzeek zzzeek added the type-bug An unexpected behavior, bug, or error label May 8, 2025
@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

cc @JelleZijlstra

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

further issues that seem related to the same thing

make two files, one with from __future__ import annotations, the other not. the one with future acts as a library for the one that does not have them. The annotations of a subclass declared against a base class that comes from the library are also not showing up if the base has a custom metaclass

lib.py

from __future__ import annotations

from typing import TYPE_CHECKING

class SomeMeta(type):
    if TYPE_CHECKING:
        someint: float

def make_base():
    return SomeMeta("Base", (object, ), {})

then make app.py:

from lib import make_base

Base = make_base()

class MyClass(Base):
    foo: int

import inspect

print(inspect.get_annotations(MyClass))

annotations for "foo" are lost in 3.14.0b1

# python 3.14.0b1
$ ~/.venv314/bin/python app.py 
{}

# python 3.14.0a7
$ ~/.venv314a7/bin/python app.py 
{'foo': <class 'int'>}

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

also suffice to say this is a way worse issue for SQLAlchemy than the conditionals thing. we have no way to work around this

@JelleZijlstra
Copy link
Member

I think the workaround is to not use from __future__ import annotations? Going to think about this some more; especially the fist report is unexpected. I want behavior under from __future__ import annotations to remain unchanged from 3.13, but clearly I missed something.

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

That's not a workaround for us, this is the SQLAlchemy library. We use modern annotations so we need from __future__ import annotations in our code since we are supporting python 3.9 through current. our users might not use this. it's not an option for the annotations to just be lost when this directive is used in some of the source files

@JelleZijlstra
Copy link
Member

This change seems to fix your cases:

diff --git a/Lib/annotationlib.py b/Lib/annotationlib.py
index 32b85534589..5e58e0c2dcd 100644
--- a/Lib/annotationlib.py
+++ b/Lib/annotationlib.py
@@ -1047,7 +1047,10 @@ def _get_dunder_annotations(obj):
 
     Does not return a fresh dictionary.
     """
-    ann = getattr(obj, "__annotations__", None)
+    if isinstance(obj, type):
+        ann = obj.__dict__.get("__annotations__", None)
+    else:
+        ann = getattr(obj, "__annotations__", None)
     if ann is None:
         return None
 

Unfortunately this issue brings us back to a state where cls.__annotations__ is unsafe, though no more so than before 3.14. (You'll see some interesting behavior in some of your examples if you use the __annotations__ attribute directly instead of get_annotations(); discussion in https://peps.python.org/pep-0749/#annotations-and-metaclasses .)

I'll put up a PR with this change later after I've had some time to write tests. Thanks for the early feedback, and sorry I didn't test behavior with the future import enough.

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

OK so I can just vendor that code right now to try to workaround? great

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

that's ironically pretty much the code we used in python versions prior to 3.10

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

this seems to be working though I am getting better results when I am using the exact version of _get_dunder_annotations that's in 3.14.0a7:

    _BASE_GET_ANNOTATIONS = type.__dict__["__annotations__"].__get__

    def _get_dunder_annotations(obj):
        if isinstance(obj, type):
            try:
                ann = _BASE_GET_ANNOTATIONS(obj)
            except AttributeError:
                # For static types, the descriptor raises AttributeError.
                return {}
        else:
            ann = getattr(obj, "__annotations__", None)
            if ann is None:
                return {}

        if not isinstance(ann, dict):
            raise ValueError(
                f"{obj!r}.__annotations__ is neither a dict nor None"
            )
        return dict(ann)

otherwise I'm getting errors on some other classes, TypeError: '>' not supported between instances of 'builtin_function_or_method' and 'int' which it's not showing exactly what it's hitting, haven't dug into it

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

I found another issue with TypedDict, and it seems similar to these, but is different in that the __annotations__ dictionary is simply missing super "class" elements - this is unworkable because the superclasses as declared are not actually in __mro__. Going to make a separate issue, apologies if it's the same thing

@zzzeek
Copy link
Author

zzzeek commented May 8, 2025

that new report is at #133701

@picnixz picnixz added stdlib Python modules in the Lib dir 3.14 bugs and security fixes labels May 8, 2025
sqlalchemy-bot pushed a commit to sqlalchemy/sqlalchemy that referenced this issue May 9, 2025
Changes to the test suite to accommodate Python 3.14 as of version
3.14.0b1

Originally this included a major breaking change to how python 3.14
implemented :pep:`649`, however this was resolved by [1].

As of a7, greenlet is skipped due to issues in a7 and later b1
in [2].

1. the change to rewrite all conditionals in annotation related tests
   is reverted.
2. test_memusage needed an explicit set_start_method() call so that
   it can continue to use plain fork
3. unfortunately at the moment greenlet has to be re-disabled for 3.14.
4. Changes to tox overall, remove pysqlcipher which hasn't worked
   in years, etc.
5. we need to support upcoming typing-extensions also, install the beta
6. 3.14.0a7 introduces major regressions to our runtime typing
   utilities, unfortunately, it's not clear if these can be resolved
7. for 3.14.0b1, we have to vendor get_annotations to work around [3]

[1] python/cpython#130881
[2] python-greenlet/greenlet#440
[3] python/cpython#133684

py314: yes
Fixes: #12405
References: #12399
Change-Id: I8715d02fae599472dd64a2a46ccf8986239ecd99
sqlalchemy-bot pushed a commit to sqlalchemy/sqlalchemy that referenced this issue May 9, 2025
Changes to the test suite to accommodate Python 3.14 as of version
3.14.0b1
Originally this included a major breaking change to how python 3.14
implemented :pep:`649`, however this was resolved by [1].
As of a7, greenlet is skipped due to issues in a7 and later b1
in [2].
1. the change to rewrite all conditionals in annotation related tests
   is reverted.
2. test_memusage needed an explicit set_start_method() call so that
   it can continue to use plain fork
3. unfortunately at the moment greenlet has to be re-disabled for 3.14.
4. Changes to tox overall, remove pysqlcipher which hasn't worked
   in years, etc.
5. we need to support upcoming typing-extensions also, install the beta
6. 3.14.0a7 introduces major regressions to our runtime typing
   utilities, unfortunately, it's not clear if these can be resolved
7. for 3.14.0b1, we have to vendor get_annotations to work around [3]

[1] python/cpython#130881
[2] python-greenlet/greenlet#440
[3] python/cpython#133684

py314: yes
Fixes: #12405
References: #12399
Change-Id: I8715d02fae599472dd64a2a46ccf8986239ecd99
@JelleZijlstra
Copy link
Member

Thanks, did you figure out anything about the TypeError: '>' not supported between instances of 'builtin_function_or_method' and 'int' thing?

@zzzeek
Copy link
Author

zzzeek commented May 9, 2025

I'm not able to get back to that same error and I suspect it had something to do with something internal, however, trying to find it I may have uncovered more problems here

@JelleZijlstra
Copy link
Member

Yeah we'll need to go back to the _BASE_GET_ANNOTATIONS hack. I'll work on it over the next few days, I want to add a similar set of test cases as in #133772 for different combinations of with/without the future.

@zzzeek
Copy link
Author

zzzeek commented May 9, 2025

OK the subsequent problem I'm having has to do with something related but out of the immediate scope here, I'm attempting to runtime set annotations on a class to something other than what they were, for the purposes of getting python's dataclass() function to interpret the class in a certain way, and if that class is in a hierarchy, the assignment of new annotations is not working. Admittedly setting annotations to something new is a little more of a hack, so something broke with that as of 3.14b1, so ill try to figure that out more.

@JelleZijlstra
Copy link
Member

That sounds like it should still work, if you can minimize the repro case please open a new issue. Thanks for all the testing!

@zzzeek
Copy link
Author

zzzeek commented May 9, 2025

if you just assign to __annotations__ that seems to have changed in 3.14.0b1, it has no effect now. there seems to be a new attribute __annotations_cache__ that's where the assignment goes but it's not used

@JelleZijlstra
Copy link
Member

Ugh that seems to be another from __future__ import annotations bug, I'll fix.

@zzzeek
Copy link
Author

zzzeek commented May 9, 2025

oh OK. I just put it in #133778 as you were typing this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3.14 bugs and security fixes stdlib Python modules in the Lib dir topic-typing type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

No branches or pull requests

4 participants