Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-142048: Fix quadratically increasing GC delays #142051

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
colesbury merged 8 commits into from
Dec 2, 2025
Merged

gh-142048: Fix quadratically increasing GC delays #142051

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
a9edb7b
reset allocation counts in gc thread-local buffers
kevmo314 Nov 28, 2025
b1a6c16
Merge branch 'main' into fix-gc-alloc-count-negative
kevmo314 Nov 30, 2025
9388518
Fix allocation count decrement and GC state update
kevmo314 Dec 1, 2025
f5e46b5
Fix formatting in gc_free_threading.c
kevmo314 Dec 1, 2025
795e703
Update Python/pystate.c
kevmo314 Dec 1, 2025
8d01b65
news and lint
kevmo314 Dec 1, 2025
8be49b5
Remove local GC allocation count flush in pystate.c
kevmo314 Dec 1, 2025
f202991
Merge branch 'main' into fix-gc-alloc-count-negative
kevmo314 Dec 1, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core_and_Builtins/2025-12-01-20-41-26.gh-issue-142048.c2YosX.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix quadratically increasing garbage collection delays in free-threaded
build.
14 changes: 13 additions & 1 deletion Python/gc_free_threading.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2210,7 +2210,19 @@ record_deallocation(PyThreadState *tstate)
gc->alloc_count--;
if (gc->alloc_count <= -LOCAL_ALLOC_COUNT_THRESHOLD) {
GCState *gcstate = &tstate->interp->gc;
_Py_atomic_add_int(&gcstate->young.count, (int)gc->alloc_count);
int count = _Py_atomic_load_int_relaxed(&gcstate->young.count);
int new_count;
do {
if (count == 0) {
break;
}
new_count = count + (int)gc->alloc_count;
if (new_count < 0) {
new_count = 0;
}
} while (!_Py_atomic_compare_exchange_int(&gcstate->young.count,
&count,
new_count));
gc->alloc_count = 0;
}
}
Expand Down
8 changes: 8 additions & 0 deletions Python/pystate.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1810,6 +1810,14 @@ tstate_delete_common(PyThreadState *tstate, int release_gil)
assert(tstate_impl->refcounts.values == NULL);
#endif

#ifdef Py_GIL_DISABLED
// Flush the thread's local GC allocation count to the global count
// before the thread state is deleted, otherwise the count is lost.
_Py_atomic_add_int(&tstate->interp->gc.young.count,
(int)tstate_impl->gc.alloc_count);
tstate_impl->gc.alloc_count = 0;
#endif

Copy link
Copy Markdown
Contributor

@colesbury colesbury Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove this for now. It's not essential for the fix and we can add it back correctly in a subsequent PR.

This is causing a thread sanitizer failure. You can see it in the job if you expand the "Display logs" section:

WARNING: ThreadSanitizer: data race (pid=19505)
  Atomic write of size 4 at 0x55bb91207c44 by thread T52:
    #0 _Py_atomic_add_int /home/runner/work/cpython/cpython/./Include/cpython/pyatomic_gcc.h:15:10 (python+0x63520e) (BuildId: c37d32af53b8782ca7323d5cb2febaf42023f6ab)
    #1 tstate_delete_common /home/runner/work/cpython/cpython/Python/pystate.c:1816:5 (python+0x63520e)
    #2 _PyThreadState_DeleteCurrent /home/runner/work/cpython/cpython/Python/pystate.c:1893:5 (python+0x635582) (BuildId: c37d32af53b8782ca7323d5cb2febaf42023f6ab)
...

  Previous write of size 4 at 0x55bb91207c44 by main thread:
    #0 gc_collect_internal /home/runner/work/cpython/cpython/Python/gc_free_threading.c:2240:33 (python+0x5940bb) (BuildId: c37d32af53b8782ca7323d5cb2febaf42023f6ab)
    #1 gc_collect_main /home/runner/work/cpython/cpython/Python/gc_free_threading.c:2414:5 (python+0x59199c) (BuildId: c37d32af53b8782ca7323d5cb2febaf42023f6ab)
    #2 _PyGC_Collect /home/runner/work/cpython/cpython/Python/gc_free_threading.c:2723:12 (python+0x591cb7) (BuildId: c37d32af53b8782ca7323d5cb2febaf42023f6ab)
...

The problem is that gc_collect_internal reads this during a stop the world pause, but this thread is no longer considered active (i.e., we have a "detached" thread state that's getting deleted).

We could move this to PyThreadState_Clear(), but let's just not include it for now. I'd like to keep the critical fix small for backporting and "losing" a few local counts probably doesn't matter.

Copy link
Copy Markdown
Contributor Author

@kevmo314 kevmo314 Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, done!

#if _Py_TIER2
_PyThreadStateImpl *_tstate = (_PyThreadStateImpl *)tstate;
if (_tstate->jit_tracer_state.code_buffer != NULL) {
Expand Down
Loading