Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 056dfc7

Browse files
carljmcarljm
authored
gh-87634: remove locking from functools.cached_property (GH-101890)
Remove the undocumented locking capabilities of functools.cached_property.
1 parent 8f64747 commit 056dfc7

5 files changed

Lines changed: 35 additions & 54 deletions

File tree

‎Doc/library/functools.rst‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -86,6 +86,14 @@ The :mod:`functools` module defines the following functions:
8686
The cached value can be cleared by deleting the attribute. This
8787
allows the *cached_property* method to run again.
8888

89+
The *cached_property* does not prevent a possible race condition in
90+
multi-threaded usage. The getter function could run more than once on the
91+
same instance, with the latest run setting the cached value. If the cached
92+
property is idempotent or otherwise not harmful to run more than once on an
93+
instance, this is fine. If synchronization is needed, implement the necessary
94+
locking inside the decorated getter function or around the cached property
95+
access.
96+
8997
Note, this decorator interferes with the operation of :pep:`412`
9098
key-sharing dictionaries. This means that instance dictionaries
9199
can take more space than usual.
@@ -110,6 +118,14 @@ The :mod:`functools` module defines the following functions:
110118
def stdev(self):
111119
return statistics.stdev(self._data)
112120

121+
122+
.. versionchanged:: 3.12
123+
Prior to Python 3.12, ``cached_property`` included an undocumented lock to
124+
ensure that in multi-threaded usage the getter function was guaranteed to
125+
run only once per instance. However, the lock was per-property, not
126+
per-instance, which could result in unacceptably high lock contention. In
127+
Python 3.12+ this locking is removed.
128+
113129
.. versionadded:: 3.8
114130

115131

‎Doc/whatsnew/3.12.rst‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -761,6 +761,15 @@ Changes in the Python API
761761
around process-global resources, which are best managed from the main interpreter.
762762
(Contributed by Dong-hee Na in :gh:`99127`.)
763763

764+
* The undocumented locking behavior of :func:`~functools.cached_property`
765+
is removed, because it locked across all instances of the class, leading to high
766+
lock contention. This means that a cached property getter function could now run
767+
more than once for a single instance, if two threads race. For most simple
768+
cached properties (e.g. those that are idempotent and simply calculate a value
769+
based on other attributes of the instance) this will be fine. If
770+
synchronization is needed, implement locking within the cached property getter
771+
function or around multi-threaded access points.
772+
764773

765774
Build Changes
766775
=============

‎Lib/functools.py‎

Lines changed: 9 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -959,15 +959,12 @@ def __isabstractmethod__(self):
959959
### cached_property() - computed once per instance, cached as attribute
960960
################################################################################
961961

962-
_NOT_FOUND = object()
963-
964962

965963
class cached_property:
966964
def __init__(self, func):
967965
self.func = func
968966
self.attrname = None
969967
self.__doc__ = func.__doc__
970-
self.lock = RLock()
971968

972969
def __set_name__(self, owner, name):
973970
if self.attrname is None:
@@ -992,21 +989,15 @@ def __get__(self, instance, owner=None):
992989
f"instance to cache {self.attrname!r} property."
993990
)
994991
raise TypeError(msg) from None
995-
val = cache.get(self.attrname, _NOT_FOUND)
996-
if val is _NOT_FOUND:
997-
with self.lock:
998-
# check if another thread filled cache while we awaited lock
999-
val = cache.get(self.attrname, _NOT_FOUND)
1000-
if val is _NOT_FOUND:
1001-
val = self.func(instance)
1002-
try:
1003-
cache[self.attrname] = val
1004-
except TypeError:
1005-
msg = (
1006-
f"The '__dict__' attribute on {type(instance).__name__!r} instance "
1007-
f"does not support item assignment for caching {self.attrname!r} property."
1008-
)
1009-
raise TypeError(msg) from None
992+
val = self.func(instance)
993+
try:
994+
cache[self.attrname] = val
995+
except TypeError:
996+
msg = (
997+
f"The '__dict__' attribute on {type(instance).__name__!r} instance "
998+
f"does not support item assignment for caching {self.attrname!r} property."
999+
)
1000+
raise TypeError(msg) from None
10101001
return val
10111002

10121003
__class_getitem__ = classmethod(GenericAlias)

‎Lib/test/test_functools.py‎

Lines changed: 0 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -2931,21 +2931,6 @@ def get_cost(self):
29312931
cached_cost = py_functools.cached_property(get_cost)
29322932

29332933

2934-
class CachedCostItemWait:
2935-
2936-
def __init__(self, event):
2937-
self._cost = 1
2938-
self.lock = py_functools.RLock()
2939-
self.event = event
2940-
2941-
@py_functools.cached_property
2942-
def cost(self):
2943-
self.event.wait(1)
2944-
with self.lock:
2945-
self._cost += 1
2946-
return self._cost
2947-
2948-
29492934
class CachedCostItemWithSlots:
29502935
__slots__ = ('_cost')
29512936

@@ -2970,27 +2955,6 @@ def test_cached_attribute_name_differs_from_func_name(self):
29702955
self.assertEqual(item.get_cost(), 4)
29712956
self.assertEqual(item.cached_cost, 3)
29722957

2973-
@threading_helper.requires_working_threading()
2974-
def test_threaded(self):
2975-
go = threading.Event()
2976-
item = CachedCostItemWait(go)
2977-
2978-
num_threads = 3
2979-
2980-
orig_si = sys.getswitchinterval()
2981-
sys.setswitchinterval(1e-6)
2982-
try:
2983-
threads = [
2984-
threading.Thread(target=lambda: item.cost)
2985-
for k in range(num_threads)
2986-
]
2987-
with threading_helper.start_threads(threads):
2988-
go.set()
2989-
finally:
2990-
sys.setswitchinterval(orig_si)
2991-
2992-
self.assertEqual(item.cost, 2)
2993-
29942958
def test_object_with_slots(self):
29952959
item = CachedCostItemWithSlots()
29962960
with self.assertRaisesRegex(

‎Misc/NEWS.d/next/Library/2023-02-13-12-55-48.gh-issue-87634.q-SBhJ.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Remove locking behavior from :func:`functools.cached_property`.

0 commit comments

Comments
 (0)