Bug report

Bug description:

Part of #127945.

ctypes C data objects have an internal pointer for what they're looking at (b_ptr). This field itself is generally fine to read non-atomically, because ctypes objects don't tend to overwrite the pointer that they're pointing to, but reading and writing the pointer's contents (such as via memcpy) isn't thread safe. This can be seen primarily with arrays:

from threading import Thread
import ctypes

buffer = (ctypes.c_char_p * 10)()

def main():
    for i in range(100):
        buffer.value = b"hello"
        buffer[1] = b"j"

threads = [Thread(target=main) for _ in range(100)]
for thread in threads:
    thread.start()

There's really only two options here, because the lockless _Py_atomic APIs can't be used for allocations of arbitrary sizes: add a critical section around all functions touching b_ptr, or just add a PyMutex around it.

I think that a mutex is the right way to go here:

• There's no chance of re-entrancy with memcpy.
• AC doesn't work for some type slots (e.g. __call__), so they can't get wrapped with @critical_section. We'd need ugly wrapper functions.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-128182: Add per-object memory access synchronization to ctypes #128490
• gh-128182: add critical section to _ctypes.Simple getters and setters #132081
• gh-128182: add critical section to _ctypes.PyCData getters and setters #132082
• gh-128182: Switch ctypes to solely critical sections #132133
• gh-128182: add critical sections to ctypes arrays getters and setters #132152