From ef27dbe8c92cd804fa8cd84d7e27a0da5fd0e105 Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Fri, 23 Aug 2024 14:44:03 +0100 Subject: [PATCH 1/4] Update what's new --- Doc/whatsnew/3.13.rst | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index c5e11458733508..c345c1e3bb2e8f 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -493,6 +493,19 @@ Incremental garbage collection The cycle garbage collector is now incremental. This means that maximum pause times are reduced by an order of magnitude or more for larger heaps. + +There are now only two generations: young and old. +When :func:`gc.collect` is not called directly, the +GC is invoked a little less frequency. When invoked it +collects the young generation and an increment of the +old generation, instead of collecting one or more generations. + +The behavior of :func:`!gc.collect` changes slightly: + +* ``gc.collect(1)``: Performs an increment of GC, + rather than collecting generation 1. +* Other calls to :func:`!gc.collect` are unchanged. + (Contributed by Mark Shannon in :gh:`108362`.) From 3aa3ecf26329cbbf6a0ebec0343ce5176077fffa Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Fri, 23 Aug 2024 15:23:32 +0100 Subject: [PATCH 2/4] Update gc module docs and fix inconsistency in gc.get_objects --- Doc/library/gc.rst | 55 +++++++++++++++++++++++++++++++++------------- Python/gc.c | 27 +++++++++++++---------- 2 files changed, 56 insertions(+), 26 deletions(-) diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 790dfdfd00b196..6342d5812074e3 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -40,11 +40,18 @@ The :mod:`gc` module provides the following functions: .. function:: collect(generation=2) - With no arguments, run a full collection. The optional argument *generation* + Perform a collection. The optional argument *generation* may be an integer specifying which generation to collect (from 0 to 2). A - :exc:`ValueError` is raised if the generation number is invalid. The sum of + :exc:`ValueError` is raised if the generation number is invalid. The sum of collected objects and uncollectable objects is returned. + Calling ``gc.collect(0)`` will perform a GC collection on the young generation. + + Calling ``gc.collect(1)`` will perform a GC collection on the young generation + and an increment of the old generation. + + Calling ``gc.collect(2)`` or ``gc.collect()`` performs a full collection + The free lists maintained for a number of built-in types are cleared whenever a full collection or collection of the highest generation (2) is run. Not all items in some free lists may be freed due to the @@ -53,6 +60,9 @@ The :mod:`gc` module provides the following functions: The effect of calling ``gc.collect()`` while the interpreter is already performing a collection is undefined. + .. versionchanged:: 3.13 + ``generation=1`` performs an increment of collection. + .. function:: set_debug(flags) @@ -68,13 +78,20 @@ The :mod:`gc` module provides the following functions: .. function:: get_objects(generation=None) + Returns a list of all objects tracked by the collector, excluding the list - returned. If *generation* is not ``None``, return only the objects tracked by - the collector that are in that generation. + returned. If *generation* is not ``None``, return only the objects as follows: + + * 0: All objects in the young generation + * 1: No objects, as there is no generation 1 + * 2: All objects in the old generation .. versionchanged:: 3.8 New *generation* parameter. + .. versionchanged:: 3.13 + Generation 1 is removed + .. audit-event:: gc.get_objects generation gc.get_objects .. function:: get_stats() @@ -101,19 +118,27 @@ The :mod:`gc` module provides the following functions: Set the garbage collection thresholds (the collection frequency). Setting *threshold0* to zero disables collection. - The GC classifies objects into three generations depending on how many - collection sweeps they have survived. New objects are placed in the youngest - generation (generation ``0``). If an object survives a collection it is moved - into the next older generation. Since generation ``2`` is the oldest - generation, objects in that generation remain there after a collection. In - order to decide when to run, the collector keeps track of the number object + The GC classifies objects into two generations depending on whether they have + survived a collection. New objects are placed in the young generation. If an + object survives a collection it is moved into the old generation. + + In order to decide when to run, the collector keeps track of the number object allocations and deallocations since the last collection. When the number of allocations minus the number of deallocations exceeds *threshold0*, collection - starts. Initially only generation ``0`` is examined. If generation ``0`` has - been examined more than *threshold1* times since generation ``1`` has been - examined, then generation ``1`` is examined as well. - With the third generation, things are a bit more complicated, - see `Collecting the oldest generation `_ for more information. + starts. For each collection, all the objects in the young generation and some + fraction of the old generation is collected. + + The fraction of the old generation that is collected is **inversely** proportional + to *threshold1*. The larger *threshold1* is, the slower objects in the old generation + are collected. + For the default value of 10, 1% of the old generation is scanned during each collection. + + *threshold2* is ignored. + + See `_ for more information. + + .. versionchanged:: 3.13 + *threshold2* is ignored .. function:: get_count() diff --git a/Python/gc.c b/Python/gc.c index f920743b9213de..e149a99444b031 100644 --- a/Python/gc.c +++ b/Python/gc.c @@ -1709,20 +1709,25 @@ _PyGC_GetObjects(PyInterpreterState *interp, int generation) GCState *gcstate = &interp->gc; PyObject *result = PyList_New(0); - if (result == NULL) { - return NULL; + /* Generation: + * -1: Return all objects + * 0: All young objects + * 1: No objects + * 2: All old objects + */ + if (result == NULL || generation == 1) { + return result; } - - if (generation == -1) { - /* If generation is -1, get all objects from all generations */ - for (int i = 0; i < NUM_GENERATIONS; i++) { - if (append_objects(result, GEN_HEAD(gcstate, i))) { - goto error; - } + if (generation <= 0) { + if (append_objects(result, &gcstate->young.head)) { + goto error; } } - else { - if (append_objects(result, GEN_HEAD(gcstate, generation))) { + if (generation != 0) { + if (append_objects(result, &gcstate->old[0].head)) { + goto error; + } + if (append_objects(result, &gcstate->old[1].head)) { goto error; } } From 280f206ee818991cb7c1b6e1b53b5e71f81961cd Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Fri, 23 Aug 2024 17:03:04 +0100 Subject: [PATCH 3/4] Apply suggestions from code review Co-authored-by: Michael Droettboom --- Doc/library/gc.rst | 4 ++-- Doc/whatsnew/3.13.rst | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index 6342d5812074e3..b4bbc310059ed5 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -83,7 +83,7 @@ The :mod:`gc` module provides the following functions: returned. If *generation* is not ``None``, return only the objects as follows: * 0: All objects in the young generation - * 1: No objects, as there is no generation 1 + * 1: No objects, as there is no generation 1 (as of Python 3.13) * 2: All objects in the old generation .. versionchanged:: 3.8 @@ -122,7 +122,7 @@ The :mod:`gc` module provides the following functions: survived a collection. New objects are placed in the young generation. If an object survives a collection it is moved into the old generation. - In order to decide when to run, the collector keeps track of the number object + In order to decide when to run, the collector keeps track of the number of object allocations and deallocations since the last collection. When the number of allocations minus the number of deallocations exceeds *threshold0*, collection starts. For each collection, all the objects in the young generation and some diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index c345c1e3bb2e8f..320c987cda4840 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -496,7 +496,7 @@ by an order of magnitude or more for larger heaps. There are now only two generations: young and old. When :func:`gc.collect` is not called directly, the -GC is invoked a little less frequency. When invoked it +GC is invoked a little less frequently. When invoked, it collects the young generation and an increment of the old generation, instead of collecting one or more generations. From bb6a60836c02cc5c61491f881e153747b260f98d Mon Sep 17 00:00:00 2001 From: Mark Shannon Date: Fri, 23 Aug 2024 17:07:06 +0100 Subject: [PATCH 4/4] Fix link --- Doc/library/gc.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst index b4bbc310059ed5..1065ec30802841 100644 --- a/Doc/library/gc.rst +++ b/Doc/library/gc.rst @@ -135,7 +135,7 @@ The :mod:`gc` module provides the following functions: *threshold2* is ignored. - See `_ for more information. + See `Garbage collector design `_ for more information. .. versionchanged:: 3.13 *threshold2* is ignored