Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit acaef71

Browse files
iritkatrieliritkatriel
authored
Merge branch 'main' into adaptive
2 parents 3bbf6ee + 67b18a1 commit acaef71

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

59 files changed

+988
-436
lines changed

‎Doc/conf.py

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -78,6 +78,13 @@
7878
.. |python_version_literal| replace:: ``Python {version}``
7979
.. |python_x_dot_y_literal| replace:: ``python{version}``
8080
.. |usr_local_bin_python_x_dot_y_literal| replace:: ``/usr/local/bin/python{version}``
81+
82+
.. Apparently this how you hack together a formatted link:
83+
(https://www.docutils.org/docs/ref/rst/directives.html#replacement-text)
84+
.. |FORCE_COLOR| replace:: ``FORCE_COLOR``
85+
.. _FORCE_COLOR: https://force-color.org/
86+
.. |NO_COLOR| replace:: ``NO_COLOR``
87+
.. _NO_COLOR: https://no-color.org/
8188
"""
8289

8390
# There are two options for replacing |today|. Either, you set today to some

‎Doc/howto/gdb_helpers.rst

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -180,7 +180,7 @@ regular machine-level integer::
180180
(gdb) p some_python_integer
181181
$4 = 42
182182

183-
The internal structure can be revealed with a cast to :c:expr:`PyLongObject *`:
183+
The internal structure can be revealed with a cast to :c:expr:`PyLongObject *`::
184184

185185
(gdb) p *(PyLongObject*)some_python_integer
186186
$5 = {ob_base = {ob_base = {ob_refcnt = 8, ob_type = 0x3dad39f5e0}, ob_size = 1},

‎Doc/library/doctest.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -136,6 +136,10 @@ examples of doctests in the standard Python test suite and libraries.
136136
Especially useful examples can be found in the standard test file
137137
:file:`Lib/test/test_doctest/test_doctest.py`.
138138

139+
.. versionadded:: 3.13
140+
Output is colorized by default and can be
141+
:ref:`controlled using environment variables <using-on-controlling-color>`.
142+
139143

140144
.. _doctest-simple-testmod:
141145

‎Doc/library/pathlib.rst

Lines changed: 0 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -1289,35 +1289,6 @@ Reading directories
12891289
raised.
12901290

12911291

1292-
.. method:: Path.scandir()
1293-
1294-
When the path points to a directory, return an iterator of
1295-
:class:`os.DirEntry` objects corresponding to entries in the directory. The
1296-
returned iterator supports the :term:`context manager` protocol. It is
1297-
implemented using :func:`os.scandir` and gives the same guarantees.
1298-
1299-
Using :meth:`~Path.scandir` instead of :meth:`~Path.iterdir` can
1300-
significantly increase the performance of code that also needs file type or
1301-
file attribute information, because :class:`os.DirEntry` objects expose
1302-
this information if the operating system provides it when scanning a
1303-
directory.
1304-
1305-
The following example displays the names of subdirectories. The
1306-
``entry.is_dir()`` check will generally not make an additional system call::
1307-
1308-
>>> p = Path('docs')
1309-
>>> with p.scandir() as entries:
1310-
... for entry in entries:
1311-
... if entry.is_dir():
1312-
... entry.name
1313-
...
1314-
'_templates'
1315-
'_build'
1316-
'_static'
1317-
1318-
.. versionadded:: 3.14
1319-
1320-
13211292
.. method:: Path.glob(pattern, *, case_sensitive=None, recurse_symlinks=False)
13221293

13231294
Glob the given relative *pattern* in the directory represented by this path,

‎Doc/library/traceback.rst

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,10 @@ The module's API can be divided into two parts:
4444
necessary for later formatting without holding references to actual exception
4545
and traceback objects.
4646

47+
.. versionadded:: 3.13
48+
Output is colorized by default and can be
49+
:ref:`controlled using environment variables <using-on-controlling-color>`.
50+
4751

4852
Module-Level Functions
4953
----------------------

‎Doc/library/unittest.rst

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -46,7 +46,6 @@ test runner
4646
a textual interface, or return a special value to indicate the results of
4747
executing the tests.
4848

49-
5049
.. seealso::
5150

5251
Module :mod:`doctest`
@@ -198,6 +197,9 @@ For a list of all the command-line options::
198197
In earlier versions it was only possible to run individual test methods and
199198
not modules or classes.
200199

200+
.. versionadded:: 3.14
201+
Output is colorized by default and can be
202+
:ref:`controlled using environment variables <using-on-controlling-color>`.
201203

202204
Command-line options
203205
~~~~~~~~~~~~~~~~~~~~

‎Doc/tutorial/datastructures.rst

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -142,8 +142,8 @@ Using Lists as Stacks
142142

143143
The list methods make it very easy to use a list as a stack, where the last
144144
element added is the first element retrieved ("last-in, first-out"). To add an
145-
item to the top of the stack, use :meth:`!~list.append`. To retrieve an item from the
146-
top of the stack, use :meth:`!~list.pop` without an explicit index. For example::
145+
item to the top of the stack, use :meth:`!append`. To retrieve an item from the
146+
top of the stack, use :meth:`!pop` without an explicit index. For example::
147147

148148
>>> stack = [3, 4, 5]
149149
>>> stack.append(6)
@@ -340,7 +340,7 @@ The :keyword:`!del` statement
340340
=============================
341341

342342
There is a way to remove an item from a list given its index instead of its
343-
value: the :keyword:`del` statement. This differs from the :meth:`!~list.pop` method
343+
value: the :keyword:`del` statement. This differs from the :meth:`!pop` method
344344
which returns a value. The :keyword:`!del` statement can also be used to remove
345345
slices from a list or clear the entire list (which we did earlier by assignment
346346
of an empty list to the slice). For example::
@@ -500,8 +500,8 @@ any immutable type; strings and numbers can always be keys. Tuples can be used
500500
as keys if they contain only strings, numbers, or tuples; if a tuple contains
501501
any mutable object either directly or indirectly, it cannot be used as a key.
502502
You can't use lists as keys, since lists can be modified in place using index
503-
assignments, slice assignments, or methods like :meth:`!~list.append` and
504-
:meth:`!~list.extend`.
503+
assignments, slice assignments, or methods like :meth:`!append` and
504+
:meth:`!extend`.
505505

506506
It is best to think of a dictionary as a set of *key: value* pairs,
507507
with the requirement that the keys are unique (within one dictionary). A pair of

‎Doc/using/cmdline.rst

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -663,14 +663,6 @@ output. To control the color output only in the Python interpreter, the
663663
precedence over ``NO_COLOR``, which in turn takes precedence over
664664
``FORCE_COLOR``.
665665

666-
.. Apparently this how you hack together a formatted link:
667-
668-
.. |FORCE_COLOR| replace:: ``FORCE_COLOR``
669-
.. _FORCE_COLOR: https://force-color.org/
670-
671-
.. |NO_COLOR| replace:: ``NO_COLOR``
672-
.. _NO_COLOR: https://no-color.org/
673-
674666
Options you shouldn't use
675667
~~~~~~~~~~~~~~~~~~~~~~~~~
676668

‎Doc/whatsnew/3.13.rst

Lines changed: 0 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -252,15 +252,6 @@ Improved error messages
252252
the canonical |NO_COLOR|_ and |FORCE_COLOR|_ environment variables.
253253
(Contributed by Pablo Galindo Salgado in :gh:`112730`.)
254254

255-
.. Apparently this how you hack together a formatted link:
256-
(https://www.docutils.org/docs/ref/rst/directives.html#replacement-text)
257-
258-
.. |FORCE_COLOR| replace:: ``FORCE_COLOR``
259-
.. _FORCE_COLOR: https://force-color.org/
260-
261-
.. |NO_COLOR| replace:: ``NO_COLOR``
262-
.. _NO_COLOR: https://no-color.org/
263-
264255
* A common mistake is to write a script with the same name as a
265256
standard library module. When this results in errors, we now
266257
display a more helpful error message:

‎Doc/whatsnew/3.14.rst

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -532,12 +532,6 @@ pathlib
532532

533533
(Contributed by Barney Gale in :gh:`73991`.)
534534

535-
* Add :meth:`pathlib.Path.scandir` to scan a directory and return an iterator
536-
of :class:`os.DirEntry` objects. This is exactly equivalent to calling
537-
:func:`os.scandir` on a path object.
538-
539-
(Contributed by Barney Gale in :gh:`125413`.)
540-
541535

542536
pdb
543537
---
@@ -616,6 +610,13 @@ unicodedata
616610
unittest
617611
--------
618612

613+
* :mod:`unittest` output is now colored by default.
614+
This can be controlled via the :envvar:`PYTHON_COLORS` environment
615+
variable as well as the canonical |NO_COLOR|_
616+
and |FORCE_COLOR|_ environment variables.
617+
See also :ref:`using-on-controlling-color`.
618+
(Contributed by Hugo van Kemenade in :gh:`127221`.)
619+
619620
* unittest discovery supports :term:`namespace package` as start
620621
directory again. It was removed in Python 3.11.
621622
(Contributed by Jacob Walls in :gh:`80958`.)

‎Include/internal/pycore_object.h

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -697,8 +697,52 @@ _PyObject_SetMaybeWeakref(PyObject *op)
697697
}
698698
}
699699

700+
extern int _PyObject_ResurrectEndSlow(PyObject *op);
700701
#endif
701702

703+
// Temporarily resurrects an object during deallocation. The refcount is set
704+
// to one.
705+
static inline void
706+
_PyObject_ResurrectStart(PyObject *op)
707+
{
708+
assert(Py_REFCNT(op) == 0);
709+
#ifdef Py_REF_DEBUG
710+
_Py_IncRefTotal(_PyThreadState_GET());
711+
#endif
712+
#ifdef Py_GIL_DISABLED
713+
_Py_atomic_store_uintptr_relaxed(&op->ob_tid, _Py_ThreadId());
714+
_Py_atomic_store_uint32_relaxed(&op->ob_ref_local, 1);
715+
_Py_atomic_store_ssize_relaxed(&op->ob_ref_shared, 0);
716+
#else
717+
Py_SET_REFCNT(op, 1);
718+
#endif
719+
}
720+
721+
// Undoes an object resurrection by decrementing the refcount without calling
722+
// _Py_Dealloc(). Returns 0 if the object is dead (the normal case), and
723+
// deallocation should continue. Returns 1 if the object is still alive.
724+
static inline int
725+
_PyObject_ResurrectEnd(PyObject *op)
726+
{
727+
#ifdef Py_REF_DEBUG
728+
_Py_DecRefTotal(_PyThreadState_GET());
729+
#endif
730+
#ifndef Py_GIL_DISABLED
731+
Py_SET_REFCNT(op, Py_REFCNT(op) - 1);
732+
return Py_REFCNT(op) != 0;
733+
#else
734+
uint32_t local = _Py_atomic_load_uint32_relaxed(&op->ob_ref_local);
735+
Py_ssize_t shared = _Py_atomic_load_ssize_acquire(&op->ob_ref_shared);
736+
if (_Py_IsOwnedByCurrentThread(op) && local == 1 && shared == 0) {
737+
// Fast-path: object has a single refcount and is owned by this thread
738+
_Py_atomic_store_uint32_relaxed(&op->ob_ref_local, 0);
739+
return 0;
740+
}
741+
// Slow-path: object has a shared refcount or is not owned by this thread
742+
return _PyObject_ResurrectEndSlow(op);
743+
#endif
744+
}
745+
702746
/* Tries to incref op and returns 1 if successful or 0 otherwise. */
703747
static inline int
704748
_Py_TryIncref(PyObject *op)

‎Include/internal/pycore_pystate.h

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -190,10 +190,18 @@ static inline void
190190
_Py_EnsureFuncTstateNotNULL(const char *func, PyThreadState *tstate)
191191
{
192192
if (tstate == NULL) {
193+
#ifndef Py_GIL_DISABLED
193194
_Py_FatalErrorFunc(func,
194195
"the function must be called with the GIL held, "
195196
"after Python initialization and before Python finalization, "
196197
"but the GIL is released (the current Python thread state is NULL)");
198+
#else
199+
_Py_FatalErrorFunc(func,
200+
"the function must be called with an active thread state, "
201+
"after Python initialization and before Python finalization, "
202+
"but it was called without an active thread state. "
203+
"Are you trying to call the C API inside of a Py_BEGIN_ALLOW_THREADS block?");
204+
#endif
197205
}
198206
}
199207

‎InternalDocs/garbage_collector.md

Lines changed: 44 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -199,22 +199,22 @@ unreachable:
199199

200200
```pycon
201201
>>> import gc
202-
>>>
202+
>>>
203203
>>> class Link:
204204
... def __init__(self, next_link=None):
205205
... self.next_link = next_link
206-
...
206+
...
207207
>>> link_3 = Link()
208208
>>> link_2 = Link(link_3)
209209
>>> link_1 = Link(link_2)
210210
>>> link_3.next_link = link_1
211211
>>> A = link_1
212212
>>> del link_1, link_2, link_3
213-
>>>
213+
>>>
214214
>>> link_4 = Link()
215215
>>> link_4.next_link = link_4
216216
>>> del link_4
217-
>>>
217+
>>>
218218
>>> # Collect the unreachable Link object (and its .__dict__ dict).
219219
>>> gc.collect()
220220
2
@@ -459,11 +459,11 @@ specifically in a generation by calling `gc.collect(generation=NUM)`.
459459
>>> # Create a reference cycle.
460460
>>> x = MyObj()
461461
>>> x.self = x
462-
>>>
462+
>>>
463463
>>> # Initially the object is in the young generation.
464464
>>> gc.get_objects(generation=0)
465465
[..., <__main__.MyObj object at 0x7fbcc12a3400>, ...]
466-
>>>
466+
>>>
467467
>>> # After a collection of the youngest generation the object
468468
>>> # moves to the old generation.
469469
>>> gc.collect(generation=0)
@@ -515,6 +515,44 @@ increment. All objects directly referred to from those stack frames are
515515
added to the working set.
516516
Then the above algorithm is repeated, starting from step 2.
517517

518+
Determining how much work to do
519+
-------------------------------
520+
521+
We need to do a certain amount of work to enusre that garbage is collected,
522+
but doing too much work slows down execution.
523+
524+
To work out how much work we need to do, consider a heap with `L` live objects
525+
and `G0` garbage objects at the start of a full scavenge and `G1` garbage objects
526+
at the end of the scavenge. We don't want the amount of garbage to grow, `G1 ≤ G0`, and
527+
we don't want too much garbage (say 1/3 of the heap maximum), `G0 ≤ L/2`.
528+
For each full scavenge we must visit all objects, `T == L + G0 + G1`, during which
529+
`G1` garbage objects are created.
530+
531+
The number of new objects created `N` must be at least the new garbage created, `N ≥ G1`,
532+
assuming that the number of live objects remains roughly constant.
533+
If we set `T == 4*N` we get `T > 4*G1` and `T = L + G0 + G1` => `L + G0 > 3G1`
534+
For a steady state heap (`G0 == G1`) we get `L > 2G0` and the desired garbage ratio.
535+
536+
In other words, to keep the garbage fraction to 1/3 or less we need to visit
537+
4 times as many objects as are newly created.
538+
539+
We can do better than this though. Not all new objects will be garbage.
540+
Consider the heap at the end of the scavenge with `L1` live objects and `G1`
541+
garbage. Also, note that `T == M + I` where `M` is the number of objects marked
542+
as reachable and `I` is the number of objects visited in increments.
543+
Everything in `M` is live, so `I ≥ G0` and in practice `I` is closer to `G0 + G1`.
544+
545+
If we choose the amount of work done such that `2*M + I == 6N` then we can do
546+
less work in most cases, but are still guaranteed to keep up.
547+
Since `I ≳ G0 + G1` (not strictly true, but close enough)
548+
`T == M + I == (6N + I)/2` and `(6N + I)/2 ≳ 4G`, so we can keep up.
549+
550+
The reason that this improves performance is that `M` is usually much larger
551+
than `I`. If `M == 10I`, then `T ≅ 3N`.
552+
553+
Finally, instead of using a fixed multiple of 8, we gradually increase it as the
554+
heap grows. This avoids wasting work for small heaps and during startup.
555+
518556

519557
Optimization: reusing fields to save memory
520558
===========================================

‎Lib/asyncio/selector_events.py

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1175,6 +1175,7 @@ def writelines(self, list_of_data):
11751175
# If the entire buffer couldn't be written, register a write handler
11761176
if self._buffer:
11771177
self._loop._add_writer(self._sock_fd, self._write_ready)
1178+
self._maybe_pause_protocol()
11781179

11791180
def can_write_eof(self):
11801181
return True

0 commit comments

Comments
 (0)