Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-41905: added abc.update_abstractmethods #22485

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Oct 6, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 21 additions & 5 deletions Doc/library/abc.rst
Original file line number Diff line number Diff line change
Expand Up @@ -174,10 +174,11 @@ The :mod:`abc` module also provides the following decorator:
to declare abstract methods for properties and descriptors.

Dynamically adding abstract methods to a class, or attempting to modify the
abstraction status of a method or class once it is created, are not
supported. The :func:`abstractmethod` only affects subclasses derived using
regular inheritance; "virtual subclasses" registered with the ABC's
:meth:`register` method are not affected.
abstraction status of a method or class once it is created, are only
supported using the :func:`update_abstractmethods` function. The
:func:`abstractmethod` only affects subclasses derived using regular
inheritance; "virtual subclasses" registered with the ABC's :meth:`register`
method are not affected.

When :func:`abstractmethod` is applied in combination with other method
descriptors, it should be applied as the innermost decorator, as shown in
Expand Down Expand Up @@ -235,7 +236,6 @@ The :mod:`abc` module also provides the following decorator:
super-call in a framework that uses cooperative
multiple-inheritance.


The :mod:`abc` module also supports the following legacy decorators:

.. decorator:: abstractclassmethod
Expand Down Expand Up @@ -335,6 +335,22 @@ The :mod:`abc` module also provides the following functions:

.. versionadded:: 3.4

.. function:: update_abstractmethods(cls)
A function to recalculate an abstract class's abstraction status. This
function should be called if a class's abstract methods have been
implemented or changed after it was created. Usually, this function should
be called from within a class decorator.

Returns *cls*, to allow usage as a class decorator.

If *cls* is not an instance of ABCMeta, does nothing.

.. note::

This function assumes that *cls*'s superclasses are already updated.
It does not update any subclasses.

.. versionadded:: 3.10

.. rubric:: Footnotes

Expand Down
7 changes: 7 additions & 0 deletions Doc/library/functools.rst
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,13 @@ The :mod:`functools` module defines the following functions:
application, implementing all six rich comparison methods instead is
likely to provide an easy speed boost.

.. note::

This decorator makes no attempt to override methods that have been
declared in the class *or its superclasses*. Meaning that if a
superclass defines a comparison operator, *total_ordering* will not
implement it again, even if the original method is abstract.

.. versionadded:: 3.2

.. versionchanged:: 3.4
Expand Down
38 changes: 38 additions & 0 deletions Lib/abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,44 @@ def _abc_caches_clear(cls):
_reset_caches(cls)


def update_abstractmethods(cls):
"""Recalculate the set of abstract methods of an abstract class.

If a class has had one of its abstract methods implemented after the
class was created, the method will not be considered implemented until
this function is called. Alternatively, if a new abstract method has been
added to the class, it will only be considered an abstract method of the
class after this function is called.

This function should be called before any use is made of the class,
usually in class decorators that add methods to the subject class.

Returns cls, to allow usage as a class decorator.

If cls is not an instance of ABCMeta, does nothing.
"""
if not hasattr(cls, '__abstractmethods__'):
# We check for __abstractmethods__ here because cls might by a C
# implementation or a python implementation (especially during
# testing), and we want to handle both cases.
return cls

abstracts = set()
# Check the existing abstract methods of the parents, keep only the ones
# that are not implemented.
for scls in cls.__bases__:
for name in getattr(scls, '__abstractmethods__', ()):
value = getattr(cls, name, None)
if getattr(value, "__isabstractmethod__", False):
abstracts.add(name)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add tests for deeper inheritance - I think iterating over __bases__ may not be enough:

>>> class A: pass
... 
>>> class B(A): pass
... 
>>> class C(B): pass
... 
>>> C.__bases__
(<class '__main__.B'>,)
>>> B.__bases__
(<class '__main__.A'>,)

Copy link
Contributor Author

@bentheiii bentheiii Oct 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually is- we are assured that the parent's __abstractmethods__ are correct since, after a class is subclassed, its abstraction status cannot change:

class A(ABC):
  @abstractmethod
  def foo(self): pass

class B(A):
  pass

# after this point, A.__abstractmethods__ will never change, so we can rely on it always being correct when checking B's Abstraction status

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a test case regardless

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, interesting. How about this case? I think you might need to break after the first hit for the method:

class A(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def foo(self):
        pass

class B(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def foo(self):
        pass

class C(A, B):
    pass

A.foo = lambda self: None


assert(C.foo is not abstract)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is expected (albeit confusing) behavior. Since abstraction status cannot change after subclassing, A is still considered abstract, even though its method is implemented. However, since namespace resolution occurs through the __dict__ attribute, The method is defined as far as C is concerned. Indeed, if you were to call update_abstractmethods(C) after this code, C would become concrete and usable, since its foo method implemented.

In essence, abstract methods shouldn't be changed after the class is used, so this use case is not handled by this implementation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, I disagree with the check for empty __subclasses__(), and I think in Irit's last example the right thing to do is to call abc.update_abstractmethods(A) and then abc.update_abstractmethods(C).

Note that it's not technically true that the abstractness cannot change once subclassed -- it may be undocumented but one could just write A.__abstractmethods__ = ().

FWIW I do agree that the code should assume that the abstractness of the bases is correct. If some base lies about it, well, Garbage In, Garbage Out.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think abc.update_abstractmethods(A) would have any impact here, A never changed.

Copy link
Member

@gvanrossum gvanrossum Oct 4, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I was commenting on your earlier example which has A.foo = lambda ... -- in that example it certainly would have an effect. :-)

My disagreement was with this quote from @bentheiii:

In essence, abstract methods shouldn't be changed after the class is used, so this use case is not handled by this implementation.

I agree with @iritkatriel that adding that test would be useful (I didn't verify -- maybe it's already added?).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I was commenting on your earlier example which has A.foo = lambda ... -- in that example it certainly would have an effect. :-)

Ah, in that case it raises an exception that A has been subclasses so you can't update it anymore. I think there's a test for that already but if not then there should be.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now - you're suggesting to remove that exception. In that case this test is probably needed.

# Also add any other newly added abstract methods.
for name, value in cls.__dict__.items():
if getattr(value, "__isabstractmethod__", False):
abstracts.add(name)
cls.__abstractmethods__ = frozenset(abstracts)
return cls


class ABC(metaclass=ABCMeta):
"""Helper class that provides a standard way to create an ABC using
inheritance.
Expand Down
3 changes: 3 additions & 0 deletions Lib/dataclasses.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import keyword
import builtins
import functools
import abc
import _thread
from types import GenericAlias

Expand Down Expand Up @@ -992,6 +993,8 @@ def _process_class(cls, init, repr, eq, order, unsafe_hash, frozen):
cls.__doc__ = (cls.__name__ +
str(inspect.signature(cls)).replace(' -> None', ''))

abc.update_abstractmethods(cls)

return cls


Expand Down
149 changes: 149 additions & 0 deletions Lib/test/test_abc.py
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,155 @@ class C(with_metaclass(abc_ABCMeta, A, B)):
pass
self.assertEqual(C.__class__, abc_ABCMeta)

def test_update_del(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def foo(self):
pass

del A.foo
self.assertEqual(A.__abstractmethods__, {'foo'})
self.assertFalse(hasattr(A, 'foo'))

abc.update_abstractmethods(A)

self.assertEqual(A.__abstractmethods__, set())
A()


def test_update_new_abstractmethods(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def bar(self):
pass

@abc.abstractmethod
def updated_foo(self):
pass

A.foo = updated_foo
abc.update_abstractmethods(A)
self.assertEqual(A.__abstractmethods__, {'foo', 'bar'})
msg = "class A with abstract methods bar, foo"
self.assertRaisesRegex(TypeError, msg, A)

def test_update_implementation(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def foo(self):
pass

class B(A):
pass

msg = "class B with abstract method foo"
self.assertRaisesRegex(TypeError, msg, B)
self.assertEqual(B.__abstractmethods__, {'foo'})

B.foo = lambda self: None

abc.update_abstractmethods(B)

B()
self.assertEqual(B.__abstractmethods__, set())

Copy link
Member

@iritkatriel iritkatriel Oct 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you need this test case to cover the first loop:

import abc
class FooABC(metaclass=abc.ABCMeta):
    @abc.abstractmethod
    def bar(self):
        pass

del FooABC.bar
assert ('bar' in FooABC.__abstractmethods__)
assert ('bar' not in FooABC.__dict__)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@iritkatriel What does that test case accomplish other than showing that FooABC is now in an inconsistent state?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gvanrossum In this comment I left out the update_abstractmethods, see Ben's version which he committed as test_update_del. Without this test the first loop in update_abstractmethods was not exercised.

def test_update_as_decorator(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def foo(self):
pass

def class_decorator(cls):
cls.foo = lambda self: None
return cls

@abc.update_abstractmethods
@class_decorator
class B(A):
pass

B()
self.assertEqual(B.__abstractmethods__, set())

def test_update_non_abc(self):
class A:
pass

@abc.abstractmethod
def updated_foo(self):
pass

A.foo = updated_foo
abc.update_abstractmethods(A)
A()
self.assertFalse(hasattr(A, '__abstractmethods__'))

def test_update_del_implementation(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def foo(self):
pass

class B(A):
def foo(self):
pass

B()

del B.foo

abc.update_abstractmethods(B)

msg = "class B with abstract method foo"
self.assertRaisesRegex(TypeError, msg, B)

def test_update_layered_implementation(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def foo(self):
pass

class B(A):
pass

class C(B):
def foo(self):
pass

C()

del C.foo

abc.update_abstractmethods(C)

msg = "class C with abstract method foo"
self.assertRaisesRegex(TypeError, msg, C)

def test_update_multi_inheritance(self):
class A(metaclass=abc_ABCMeta):
@abc.abstractmethod
def foo(self):
pass

class B(metaclass=abc_ABCMeta):
def foo(self):
pass

class C(B, A):
@abc.abstractmethod
def foo(self):
pass

self.assertEqual(C.__abstractmethods__, {'foo'})

del C.foo

abc.update_abstractmethods(C)

self.assertEqual(C.__abstractmethods__, set())

C()


class TestABCWithInitSubclass(unittest.TestCase):
def test_works_with_init_subclass(self):
Expand Down
37 changes: 37 additions & 0 deletions Lib/test/test_dataclasses.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@

from dataclasses import *

import abc
import pickle
import inspect
import builtins
Expand Down Expand Up @@ -3332,6 +3333,42 @@ class C:

## replace(c, x=5)

class TestAbstract(unittest.TestCase):
def test_abc_implementation(self):
class Ordered(abc.ABC):
@abc.abstractmethod
def __lt__(self, other):
pass

@abc.abstractmethod
def __le__(self, other):
pass

@dataclass(order=True)
class Date(Ordered):
year: int
month: 'Month'
day: 'int'

self.assertFalse(inspect.isabstract(Date))
self.assertGreater(Date(2020,12,25), Date(2020,8,31))

def test_maintain_abc(self):
class A(abc.ABC):
@abc.abstractmethod
def foo(self):
pass

@dataclass
class Date(A):
year: int
month: 'Month'
day: 'int'

self.assertTrue(inspect.isabstract(Date))
msg = 'class Date with abstract method foo'
self.assertRaisesRegex(TypeError, msg, Date)


if __name__ == '__main__':
unittest.main()
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
A new function in abc: *update_abstractmethods* to re-calculate an abstract class's abstract status. In addition, *dataclass* has been changed to call this function.