Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 940cbf9

Browse files
furkanonderfurkanonder
authored
Merge branch 'main' into gh-125620 
2 parents d2e3bb6 + c3164ae commit 940cbf9

63 files changed

Lines changed: 2205 additions & 711 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Doc/deprecations/pending-removal-in-future.rst‎

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -4,8 +4,13 @@ Pending removal in future versions
44
The following APIs will be removed in the future,
55
although there is currently no date scheduled for their removal.
66

7-
* :mod:`argparse`: Nesting argument groups and nesting mutually exclusive
8-
groups are deprecated.
7+
* :mod:`argparse`:
8+
9+
* Nesting argument groups and nesting mutually exclusive
10+
groups are deprecated.
11+
* Passing the undocumented keyword argument *prefix_chars* to
12+
:meth:`~argparse.ArgumentParser.add_argument_group` is now
13+
deprecated.
914

1015
* :mod:`array`'s ``'u'`` format code (:gh:`57281`)
1116

‎Doc/howto/argparse.rst‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -841,6 +841,53 @@ translated messages.
841841

842842
To translate your own strings in the :mod:`argparse` output, use :mod:`gettext`.
843843

844+
Custom type converters
845+
======================
846+
847+
The :mod:`argparse` module allows you to specify custom type converters for
848+
your command-line arguments. This allows you to modify user input before it's
849+
stored in the :class:`argparse.Namespace`. This can be useful when you need to
850+
pre-process the input before it is used in your program.
851+
852+
When using a custom type converter, you can use any callable that takes a
853+
single string argument (the argument value) and returns the converted value.
854+
However, if you need to handle more complex scenarios, you can use a custom
855+
action class with the **action** parameter instead.
856+
857+
For example, let's say you want to handle arguments with different prefixes and
858+
process them accordingly::
859+
860+
import argparse
861+
862+
parser = argparse.ArgumentParser(prefix_chars='-+')
863+
864+
parser.add_argument('-a', metavar='<value>', action='append',
865+
type=lambda x: ('-', x))
866+
parser.add_argument('+a', metavar='<value>', action='append',
867+
type=lambda x: ('+', x))
868+
869+
args = parser.parse_args()
870+
print(args)
871+
872+
Output:
873+
874+
.. code-block:: shell-session
875+
876+
$ python prog.py -a value1 +a value2
877+
Namespace(a=[('-', 'value1'), ('+', 'value2')])
878+
879+
In this example, we:
880+
881+
* Created a parser with custom prefix characters using the ``prefix_chars``
882+
parameter.
883+
884+
* Defined two arguments, ``-a`` and ``+a``, which used the ``type`` parameter to
885+
create custom type converters to store the value in a tuple with the prefix.
886+
887+
Without the custom type converters, the arguments would have treated the ``-a``
888+
and ``+a`` as the same argument, which would have been undesirable. By using custom
889+
type converters, we were able to differentiate between the two arguments.
890+
844891
Conclusion
845892
==========
846893

‎Doc/library/_thread.rst‎

Lines changed: 3 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -187,6 +187,9 @@ Lock objects have the following methods:
187187
.. versionchanged:: 3.2
188188
Lock acquires can now be interrupted by signals on POSIX.
189189

190+
.. versionchanged:: 3.14
191+
Lock acquires can now be interrupted by signals on Windows.
192+
190193

191194
.. method:: lock.release()
192195

@@ -219,12 +222,6 @@ In addition to these methods, lock objects can also be used via the
219222
* Calling :func:`sys.exit` or raising the :exc:`SystemExit` exception is
220223
equivalent to calling :func:`_thread.exit`.
221224

222-
* It is platform-dependent whether the :meth:`~threading.Lock.acquire` method
223-
on a lock can be interrupted (so that the :exc:`KeyboardInterrupt` exception
224-
will happen immediately, rather than only after the lock has been acquired or
225-
the operation has timed out). It can be interrupted on POSIX, but not on
226-
Windows.
227-
228225
* When the main thread exits, it is system defined whether the other threads
229226
survive. On most systems, they are killed without executing
230227
:keyword:`try` ... :keyword:`finally` clauses or executing object

‎Doc/library/argparse.rst‎

Lines changed: 31 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,8 @@ ArgumentParser objects
6161
formatter_class=argparse.HelpFormatter, \
6262
prefix_chars='-', fromfile_prefix_chars=None, \
6363
argument_default=None, conflict_handler='error', \
64-
add_help=True, allow_abbrev=True, exit_on_error=True)
64+
add_help=True, allow_abbrev=True, exit_on_error=True, \
65+
suggest_on_error=False)
6566

6667
Create a new :class:`ArgumentParser` object. All parameters should be passed
6768
as keyword arguments. Each parameter has its own more detailed description
@@ -103,6 +104,10 @@ ArgumentParser objects
103104
* exit_on_error_ - Determines whether or not ArgumentParser exits with
104105
error info when an error occurs. (default: ``True``)
105106

107+
* suggest_on_error_ - Enables suggestions for mistyped argument choices
108+
and subparser names (default: ``False``)
109+
110+
106111
.. versionchanged:: 3.5
107112
*allow_abbrev* parameter was added.
108113

@@ -559,6 +564,27 @@ If the user would like to catch errors manually, the feature can be enabled by s
559564

560565
.. versionadded:: 3.9
561566

567+
suggest_on_error
568+
^^^^^^^^^^^^^^^^
569+
570+
By default, when a user passes an invalid argument choice or subparser name,
571+
:class:`ArgumentParser` will exit with error info and list the permissible
572+
argument choices (if specified) or subparser names as part of the error message.
573+
574+
If the user would like to enable suggestions for mistyped argument choices and
575+
subparser names, the feature can be enabled by setting ``suggest_on_error`` to
576+
``True``. Note that this only applies for arguments when the choices specified
577+
are strings::
578+
579+
>>> parser = argparse.ArgumentParser(description='Process some integers.', suggest_on_error=True)
580+
>>> parser.add_argument('--action', choices=['sum', 'max'])
581+
>>> parser.add_argument('integers', metavar='N', type=int, nargs='+',
582+
... help='an integer for the accumulator')
583+
>>> parser.parse_args(['--action', 'sumn', 1, 2, 3])
584+
tester.py: error: argument --action: invalid choice: 'sumn', maybe you meant 'sum'? (choose from 'sum', 'max')
585+
586+
.. versionadded:: 3.14
587+
562588

563589
The add_argument() method
564590
-------------------------
@@ -1868,6 +1894,10 @@ Argument groups
18681894
The function exists on the API by accident through inheritance and
18691895
will be removed in the future.
18701896

1897+
.. deprecated:: 3.14
1898+
Passing prefix_chars_ to :meth:`add_argument_group`
1899+
is now deprecated.
1900+
18711901

18721902
Mutual exclusion
18731903
^^^^^^^^^^^^^^^^

‎Doc/library/asyncio-dev.rst‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -103,7 +103,8 @@ To handle signals the event loop must be
103103
run in the main thread.
104104

105105
The :meth:`loop.run_in_executor` method can be used with a
106-
:class:`concurrent.futures.ThreadPoolExecutor` to execute
106+
:class:`concurrent.futures.ThreadPoolExecutor` or
107+
:class:`~concurrent.futures.InterpreterPoolExecutor` to execute
107108
blocking code in a different OS thread without blocking the OS thread
108109
that the event loop runs in.
109110

@@ -128,7 +129,8 @@ if a function performs a CPU-intensive calculation for 1 second,
128129
all concurrent asyncio Tasks and IO operations would be delayed
129130
by 1 second.
130131

131-
An executor can be used to run a task in a different thread or even in
132+
An executor can be used to run a task in a different thread,
133+
including in a different interpreter, or even in
132134
a different process to avoid blocking the OS thread with the
133135
event loop. See the :meth:`loop.run_in_executor` method for more
134136
details.

‎Doc/library/asyncio-eventloop.rst‎

Lines changed: 8 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1305,6 +1305,12 @@ Executing code in thread or process pools
13051305
pool, cpu_bound)
13061306
print('custom process pool', result)
13071307

1308+
# 4. Run in a custom interpreter pool:
1309+
with concurrent.futures.InterpreterPoolExecutor() as pool:
1310+
result = await loop.run_in_executor(
1311+
pool, cpu_bound)
1312+
print('custom interpreter pool', result)
1313+
13081314
if __name__ == '__main__':
13091315
asyncio.run(main())
13101316

@@ -1329,7 +1335,8 @@ Executing code in thread or process pools
13291335

13301336
Set *executor* as the default executor used by :meth:`run_in_executor`.
13311337
*executor* must be an instance of
1332-
:class:`~concurrent.futures.ThreadPoolExecutor`.
1338+
:class:`~concurrent.futures.ThreadPoolExecutor`, which includes
1339+
:class:`~concurrent.futures.InterpreterPoolExecutor`.
13331340

13341341
.. versionchanged:: 3.11
13351342
*executor* must be an instance of

‎Doc/library/asyncio-llapi-index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,7 @@ See also the main documentation section about the
9696
- Invoke a callback *at* the given time.
9797

9898

99-
.. rubric:: Thread/Process Pool
99+
.. rubric:: Thread/Interpreter/Process Pool
100100
.. list-table::
101101
:widths: 50 50
102102
:class: full-width-table

‎Doc/library/concurrent.futures.rst‎

Lines changed: 131 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,9 +15,10 @@ The :mod:`concurrent.futures` module provides a high-level interface for
1515
asynchronously executing callables.
1616

1717
The asynchronous execution can be performed with threads, using
18-
:class:`ThreadPoolExecutor`, or separate processes, using
19-
:class:`ProcessPoolExecutor`. Both implement the same interface, which is
20-
defined by the abstract :class:`Executor` class.
18+
:class:`ThreadPoolExecutor` or :class:`InterpreterPoolExecutor`,
19+
or separate processes, using :class:`ProcessPoolExecutor`.
20+
Each implements the same interface, which is defined
21+
by the abstract :class:`Executor` class.
2122

2223
.. include:: ../includes/wasm-notavail.rst
2324

@@ -63,7 +64,8 @@ Executor Objects
6364
setting *chunksize* to a positive integer. For very long iterables,
6465
using a large value for *chunksize* can significantly improve
6566
performance compared to the default size of 1. With
66-
:class:`ThreadPoolExecutor`, *chunksize* has no effect.
67+
:class:`ThreadPoolExecutor` and :class:`InterpreterPoolExecutor`,
68+
*chunksize* has no effect.
6769

6870
.. versionchanged:: 3.5
6971
Added the *chunksize* argument.
@@ -227,6 +229,111 @@ ThreadPoolExecutor Example
227229
print('%r page is %d bytes' % (url, len(data)))
228230

229231

232+
InterpreterPoolExecutor
233+
-----------------------
234+
235+
The :class:`InterpreterPoolExecutor` class uses a pool of interpreters
236+
to execute calls asynchronously. It is a :class:`ThreadPoolExecutor`
237+
subclass, which means each worker is running in its own thread.
238+
The difference here is that each worker has its own interpreter,
239+
and runs each task using that interpreter.
240+
241+
The biggest benefit to using interpreters instead of only threads
242+
is true multi-core parallelism. Each interpreter has its own
243+
:term:`Global Interpreter Lock <global interpreter lock>`, so code
244+
running in one interpreter can run on one CPU core, while code in
245+
another interpreter runs unblocked on a different core.
246+
247+
The tradeoff is that writing concurrent code for use with multiple
248+
interpreters can take extra effort. However, this is because it
249+
forces you to be deliberate about how and when interpreters interact,
250+
and to be explicit about what data is shared between interpreters.
251+
This results in several benefits that help balance the extra effort,
252+
including true multi-core parallelism, For example, code written
253+
this way can make it easier to reason about concurrency. Another
254+
major benefit is that you don't have to deal with several of the
255+
big pain points of using threads, like nrace conditions.
256+
257+
Each worker's interpreter is isolated from all the other interpreters.
258+
"Isolated" means each interpreter has its own runtime state and
259+
operates completely independently. For example, if you redirect
260+
:data:`sys.stdout` in one interpreter, it will not be automatically
261+
redirected any other interpreter. If you import a module in one
262+
interpreter, it is not automatically imported in any other. You
263+
would need to import the module separately in interpreter where
264+
you need it. In fact, each module imported in an interpreter is
265+
a completely separate object from the same module in a different
266+
interpreter, including :mod:`sys`, :mod:`builtins`,
267+
and even ``__main__``.
268+
269+
Isolation means a mutable object, or other data, cannot be used
270+
by more than one interpreter at the same time. That effectively means
271+
interpreters cannot actually share such objects or data. Instead,
272+
each interpreter must have its own copy, and you will have to
273+
synchronize any changes between the copies manually. Immutable
274+
objects and data, like the builtin singletons, strings, and tuples
275+
of immutable objects, don't have these limitations.
276+
277+
Communicating and synchronizing between interpreters is most effectively
278+
done using dedicated tools, like those proposed in :pep:`734`. One less
279+
efficient alternative is to serialize with :mod:`pickle` and then send
280+
the bytes over a shared :mod:`socket <socket>` or
281+
:func:`pipe <os.pipe>`.
282+
283+
.. class:: InterpreterPoolExecutor(max_workers=None, thread_name_prefix='', initializer=None, initargs=(), shared=None)
284+
285+
A :class:`ThreadPoolExecutor` subclass that executes calls asynchronously
286+
using a pool of at most *max_workers* threads. Each thread runs
287+
tasks in its own interpreter. The worker interpreters are isolated
288+
from each other, which means each has its own runtime state and that
289+
they can't share any mutable objects or other data. Each interpreter
290+
has its own :term:`Global Interpreter Lock <global interpreter lock>`,
291+
which means code run with this executor has true multi-core parallelism.
292+
293+
The optional *initializer* and *initargs* arguments have the same
294+
meaning as for :class:`!ThreadPoolExecutor`: the initializer is run
295+
when each worker is created, though in this case it is run.in
296+
the worker's interpreter. The executor serializes the *initializer*
297+
and *initargs* using :mod:`pickle` when sending them to the worker's
298+
interpreter.
299+
300+
.. note::
301+
Functions defined in the ``__main__`` module cannot be pickled
302+
and thus cannot be used.
303+
304+
.. note::
305+
The executor may replace uncaught exceptions from *initializer*
306+
with :class:`~concurrent.futures.interpreter.ExecutionFailed`.
307+
308+
The optional *shared* argument is a :class:`dict` of objects that all
309+
interpreters in the pool share. The *shared* items are added to each
310+
interpreter's ``__main__`` module. Not all objects are shareable.
311+
Shareable objects include the builtin singletons, :class:`str`
312+
and :class:`bytes`, and :class:`memoryview`. See :pep:`734`
313+
for more info.
314+
315+
Other caveats from parent :class:`ThreadPoolExecutor` apply here.
316+
317+
:meth:`~Executor.submit` and :meth:`~Executor.map` work like normal,
318+
except the worker serializes the callable and arguments using
319+
:mod:`pickle` when sending them to its interpreter. The worker
320+
likewise serializes the return value when sending it back.
321+
322+
.. note::
323+
Functions defined in the ``__main__`` module cannot be pickled
324+
and thus cannot be used.
325+
326+
When a worker's current task raises an uncaught exception, the worker
327+
always tries to preserve the exception as-is. If that is successful
328+
then it also sets the ``__cause__`` to a corresponding
329+
:class:`~concurrent.futures.interpreter.ExecutionFailed`
330+
instance, which contains a summary of the original exception.
331+
In the uncommon case that the worker is not able to preserve the
332+
original as-is then it directly preserves the corresponding
333+
:class:`~concurrent.futures.interpreter.ExecutionFailed`
334+
instance instead.
335+
336+
230337
ProcessPoolExecutor
231338
-------------------
232339

@@ -574,6 +681,26 @@ Exception classes
574681

575682
.. versionadded:: 3.7
576683

684+
.. currentmodule:: concurrent.futures.interpreter
685+
686+
.. exception:: BrokenInterpreterPool
687+
688+
Derived from :exc:`~concurrent.futures.thread.BrokenThreadPool`,
689+
this exception class is raised when one of the workers
690+
of a :class:`~concurrent.futures.InterpreterPoolExecutor`
691+
has failed initializing.
692+
693+
.. versionadded:: next
694+
695+
.. exception:: ExecutionFailed
696+
697+
Raised from :class:`~concurrent.futures.InterpreterPoolExecutor` when
698+
the given initializer fails or from
699+
:meth:`~concurrent.futures.Executor.submit` when there's an uncaught
700+
exception from the submitted task.
701+
702+
.. versionadded:: next
703+
577704
.. currentmodule:: concurrent.futures.process
578705

579706
.. exception:: BrokenProcessPool

‎Doc/library/socket.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -451,8 +451,8 @@ Constants
451451
network interface instead of its name.
452452

453453
.. versionchanged:: 3.14
454-
Added missing ``IP_RECVERR``, ``IP_RECVTTL``, and ``IP_RECVORIGDSTADDR``
455-
on Linux.
454+
Added missing ``IP_RECVERR``, ``IPV6_RECVERR``, ``IP_RECVTTL``, and
455+
``IP_RECVORIGDSTADDR`` on Linux.
456456

457457
.. versionchanged:: 3.14
458458
Added support for ``TCP_QUICKACK`` on Windows platforms when available.

‎Doc/library/threading.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -567,6 +567,9 @@ All methods are executed atomically.
567567
Lock acquisition can now be interrupted by signals on POSIX if the
568568
underlying threading implementation supports it.
569569

570+
.. versionchanged:: 3.14
571+
Lock acquisition can now be interrupted by signals on Windows.
572+
570573

571574
.. method:: release()
572575

0 commit comments

Comments
 (0)