Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-139103: fix free-threading dataclass.__init__ perf issue #141596

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
colesbury merged 9 commits into from
Nov 19, 2025
+25 −0
Merged

gh-139103: fix free-threading dataclass.__init__ perf issue #141596

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
fbda041
make lock with fine-grianed
LindaSummer Nov 15, 2025
273d2af
add benchmark script
LindaSummer Nov 15, 2025
724897b
add news entry.
LindaSummer Nov 15, 2025
e915039
fix for pre-commit.
LindaSummer Nov 15, 2025
65c695f
fix news for docs check.
LindaSummer Nov 15, 2025
aaaec25
update news entry
LindaSummer Nov 16, 2025
c02223e
remove redundant lock
LindaSummer Nov 17, 2025
34d4ffa
Format if statement and add comment
colesbury Nov 18, 2025
3846637
Merge branch 'main' into fix/dataclass-perf
LindaSummer Nov 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2025-11-15-23-58-23.gh-issue-139103.9cVYJ0.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Make ``type_setattro`` use defer refcount in free-threading for functions without defer refcount.
Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is too technical for a news entry. Instead, can we say something like "Improve multithreaded scaling of dataclasses on the free-threaded build"?

Copy link
Copy Markdown
Contributor Author

@LindaSummer LindaSummer Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ZeroIntensity ,

Thanks very much for your suggestion!
I will update the news entry as suggested.

Best Regards,
Edward

13 changes: 13 additions & 0 deletions Objects/typeobject.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6546,6 +6546,18 @@ type_setattro(PyObject *self, PyObject *name, PyObject *value)
assert(!_PyType_HasFeature(metatype, Py_TPFLAGS_INLINE_VALUES));
assert(!_PyType_HasFeature(metatype, Py_TPFLAGS_MANAGED_DICT));

#ifdef Py_GIL_DISABLED
if (value != NULL && PyFunction_Check(value)) {
if (!_PyObject_HasDeferredRefcount(value)) {
BEGIN_TYPE_LOCK();
Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't need to hold the type lock here, since PyUnstable_Object_EnableDeferredRefcount is thread-safe.

Copy link
Copy Markdown
Contributor Author

@LindaSummer LindaSummer Nov 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ZeroIntensity ,

Thanks very much for pointing this out. ❤

I go through the code of PyUnstable_Object_EnableDeferredRefcount and find that it's thread-safe.

cpython/Objects/object.c

Lines 2734 to 2739 in 3d14805

if (_Py_atomic_compare_exchange_uint8(&op->ob_gc_bits, &bits, bits | _PyGC_BITS_DEFERRED) == 0)
{
// Someone beat us to it!
return 0;
}
_Py_atomic_add_ssize(&op->ob_ref_shared, _Py_REF_SHARED(_Py_REF_DEFERRED, 0));

It uses compare-and-set to guard the _PyGC_BITS_DEFERRED bit flag before updating the payload of the deferred refcount. So only one thread could make this change.

I removed the TYPE_LOCK and the double check of the _PyObject_HasDeferredRefcount. They are unnecessary.

Best Regards,
Edward

if (!_PyObject_HasDeferredRefcount(value)) {
Copy link
Copy Markdown
Member

@ZeroIntensity ZeroIntensity Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this checked twice?

Copy link
Copy Markdown
Contributor Author

@LindaSummer LindaSummer Nov 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @ZeroIntensity ,

I tried to double-check the flag to reduce the conflict of entering the lock.
But as you mentioned above, we don't need a lock for PyUnstable_Object_EnableDeferredRefcount .
This double check has been removed with the lock.

An atomic action on the type flag is very efficient and is safe for our scenario.

Thanks very much for your suggestion!

Best Regards,
Edward

PyUnstable_Object_EnableDeferredRefcount(value);
}
END_TYPE_LOCK();
}
}
#endif

PyObject *old_value = NULL;
PyObject *descr = _PyType_LookupRef(metatype, name);
if (descr != NULL) {
Expand Down Expand Up @@ -6573,6 +6585,7 @@ type_setattro(PyObject *self, PyObject *name, PyObject *value)
}
}


Comment thread
LindaSummer marked this conversation as resolved.
Outdated
BEGIN_TYPE_DICT_LOCK(dict);
res = type_update_dict(type, (PyDictObject *)dict, name, value, &old_value);
assert(_PyType_CheckConsistency(type));
Expand Down
12 changes: 12 additions & 0 deletions Tools/ftscalingbench/ftscalingbench.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@
import sys
import threading
import time
from dataclasses import dataclass
from operator import methodcaller

# The iterations in individual benchmarks are scaled by this factor.
Expand Down Expand Up @@ -202,6 +203,17 @@ def method_caller():
for i in range(1000 * WORK_SCALE):
mc(obj)

@dataclass
class MyDataClass:
x: int
y: int
z: int

@register_benchmark
def instantiate_dataclass():
for _ in range(1000 * WORK_SCALE):
obj = MyDataClass(x=1, y=2, z=3)

def bench_one_thread(func):
t0 = time.perf_counter_ns()
func()
Expand Down
Loading