Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Some pre-finalization callbacks can create other callbacks #136003

Open
@ZeroIntensity

Description

@ZeroIntensity

Bug report

Bug description:

Currently, "pre-finalization" (callbacks executed while the interpreter is still fully intact) looks like this:

cpython/Python/pylifecycle.c

Lines 2023 to 2038 in 642e5df

wait_for_thread_shutdown(tstate);
// Make any remaining pending calls.
_Py_FinishPendingCalls(tstate);
/* The interpreter is still entirely intact at this point, and the
* exit funcs may be relying on that. In particular, if some thread
* or exit func is still waiting to do an import, the import machinery
* expects Py_IsInitialized() to return true. So don't say the
* runtime is uninitialized until after the exit funcs have run.
* Note that Threading.py uses an exit func to do a join on all the
* threads created thru it, so this also protects pending imports in
* the threads created via Threading.
*/
_PyAtExit_Call(tstate->interp);

Threads are joined first, then pending calls are executed, and finally atexit callbacks are executed. The problem is that any of these three can create one another, such as a pending call creating a thread, or an atexit handler scheduling a pending call. The best way to demonstrate this is by creating a new thread inside of an atexit callback:

import atexit
import threading
import time


def run():
    print(24)
    time.sleep(1)
    print(42)

@atexit.register
def start_thread():
    threading.Thread(target=run).start()

On 3.13+ (and probably earlier), you'll get one of these three outputs:

  1. 24 is printed.
  2. Nothing is printed.
  3. A fatal error occurs due to the stdout lock being unavailable.

This is because the thread created is incorrectly non-daemon, because the call to threading._shutdown (which should join it) has already happened.

@ericsnowcurrently, I discussed this with you at the PyCon sprints. I think we came to the agreement that the best way to fix this is by looping until all three of them execute no callbacks.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Metadata

Metadata

Assignees

No one assigned

    Labels

    interpreter-core(Objects, Python, Grammar, and Parser dirs)topic-C-APItype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions