gh-114756: Update FAQ section on removing the GIL (#114957)

da-woods · aisk · web-flow · commit 0e2ab73dc31e · 2024-02-06T10:55:44.000-05:00
Update FAQ section on removing the GIL to reflect recent progress on PEP 703 and PEP 684.

Co-authored-by: AN Long &lt;aisk@users.noreply.github.com&gt;
diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst
@@ -405,22 +405,37 @@ lists.  When in doubt, use a mutex!
 Can't we get rid of the Global Interpreter Lock?
 ------------------------------------------------
 
-.. XXX link to dbeazley's talk about GIL?
-
 The :term:`global interpreter lock` (GIL) is often seen as a hindrance to Python's
 deployment on high-end multiprocessor server machines, because a multi-threaded
 Python program effectively only uses one CPU, due to the insistence that
 (almost) all Python code can only run while the GIL is held.
 
-Back in the days of Python 1.5, Greg Stein actually implemented a comprehensive
+With the approval of :pep:`703` work is now underway to remove the GIL from the
+CPython implementation of Python.  Initially it will be implemented as an
+optional compiler flag when building the interpreter, and so separate
+builds will be available with and without the GIL.  Long-term, the hope is
+to settle on a single build, once the performance implications of removing the
+GIL are fully understood.  Python 3.13 is likely to be the first release
+containing this work, although it may not be completely functional in this
+release.
+
+The current work to remove the GIL is based on a
+`fork of Python 3.9 with the GIL removed <https://github.com/colesbury/nogil>`_
+by Sam Gross.
+Prior to that,
+in the days of Python 1.5, Greg Stein actually implemented a comprehensive
 patch set (the "free threading" patches) that removed the GIL and replaced it
-with fine-grained locking.  Adam Olsen recently did a similar experiment
+with fine-grained locking.  Adam Olsen did a similar experiment
 in his `python-safethread <https://code.google.com/archive/p/python-safethread>`_
-project.  Unfortunately, both experiments exhibited a sharp drop in single-thread
+project.  Unfortunately, both of these earlier experiments exhibited a sharp
+drop in single-thread
 performance (at least 30% slower), due to the amount of fine-grained locking
-necessary to compensate for the removal of the GIL.
+necessary to compensate for the removal of the GIL.  The Python 3.9 fork
+is the first attempt at removing the GIL with an acceptable performance
+impact.
 
-This doesn't mean that you can't make good use of Python on multi-CPU machines!
+The presence of the GIL in current Python releases
+doesn't mean that you can't make good use of Python on multi-CPU machines!
 You just have to be creative with dividing the work up between multiple
 *processes* rather than multiple *threads*.  The
 :class:`~concurrent.futures.ProcessPoolExecutor` class in the new
@@ -434,22 +449,13 @@ thread of execution is in the C code and allow other threads to get some work
 done.  Some standard library modules such as :mod:`zlib` and :mod:`hashlib`
 already do this.
 
-It has been suggested that the GIL should be a per-interpreter-state lock rather
-than truly global; interpreters then wouldn't be able to share objects.
-Unfortunately, this isn't likely to happen either.  It would be a tremendous
-amount of work, because many object implementations currently have global state.
-For example, small integers and short strings are cached; these caches would
-have to be moved to the interpreter state.  Other object types have their own
-free list; these free lists would have to be moved to the interpreter state.
-And so on.
-
-And I doubt that it can even be done in finite time, because the same problem
-exists for 3rd party extensions.  It is likely that 3rd party extensions are
-being written at a faster rate than you can convert them to store all their
-global state in the interpreter state.
-
-And finally, once you have multiple interpreters not sharing any state, what
-have you gained over running each interpreter in a separate process?
+An alternative approach to reducing the impact of the GIL is
+to make the GIL a per-interpreter-state lock rather than truly global.
+This was :ref:`first implemented in Python 3.12 <whatsnew312-pep684>` and is
+available in the C API. A Python interface to it is expected in Python 3.13.
+The main limitation to it at the moment is likely to be 3rd party extension
+modules, since these must be written with multiple interpreters in mind in
+order to be usable, so many older extension modules will not be usable.
 
 
 Input and Output
