Thanks to visit codestin.com
Credit goes to github.com

Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
5c69d38
Make ThreadPoolExecutor extensible.
ericsnowcurrently Sep 25, 2024
01789be
Add InterpreterPoolExecutor.
ericsnowcurrently Sep 25, 2024
6def4be
Clean up the interpreter if initialize() fails.
ericsnowcurrently Sep 27, 2024
84993a5
Add a missing import.
ericsnowcurrently Sep 27, 2024
c540cf0
Fix some typos.
ericsnowcurrently Sep 27, 2024
45d584d
Add more tests.
ericsnowcurrently Sep 27, 2024
c90c016
Add docs.
ericsnowcurrently Sep 27, 2024
1cb4657
Add a NEwS entry.
ericsnowcurrently Sep 27, 2024
4dc0989
Fix the last test.
ericsnowcurrently Sep 27, 2024
57b2db6
Add more tests.
ericsnowcurrently Sep 27, 2024
75e11d2
Simplify ExecutionFailed.
ericsnowcurrently Sep 30, 2024
69c2b8e
Fix the signature of resolve_task().
ericsnowcurrently Sep 30, 2024
f03c314
Capture any uncaught exception.
ericsnowcurrently Sep 30, 2024
4806d9f
Add TODO comments.
ericsnowcurrently Sep 30, 2024
efc0395
Docs fixes.
ericsnowcurrently Sep 30, 2024
a29aee3
Automatically apply textwrap.dedent() to scripts.
ericsnowcurrently Sep 30, 2024
8bab457
Fix the WASI build.
ericsnowcurrently Sep 30, 2024
cd29914
wasi
ericsnowcurrently Oct 1, 2024
0287f3b
Ignore race in test.
ericsnowcurrently Oct 1, 2024
80cd7b1
Add BrokenInterpreterPool.
ericsnowcurrently Oct 8, 2024
f8d4273
Tweak the docs.
ericsnowcurrently Oct 8, 2024
3a8bfce
Clarify the InterpreterPoolExecutor docs.
ericsnowcurrently Oct 8, 2024
af6c27a
Catch all exceptions.
ericsnowcurrently Oct 8, 2024
8c0a405
Factor out exception serialization helpers.
ericsnowcurrently Oct 8, 2024
1ae7ca2
Set the ExecutionFailed error as __cause__.
ericsnowcurrently Oct 8, 2024
d24e85d
Drop the exception serialization helpers.
ericsnowcurrently Oct 8, 2024
05a03ad
Always finalize if there is an error in initialize().
ericsnowcurrently Oct 8, 2024
f150931
Explicitly note the problem with functions defined in __main__.
ericsnowcurrently Oct 8, 2024
97d0292
Handle the case where interpreters.queues doesn't exist.
ericsnowcurrently Oct 8, 2024
baf0504
Merge branch 'main' into interpreter-pool-executor
ericsnowcurrently Oct 15, 2024
5c3a327
Add a What's New entry about InterpreterPoolExecutor.
ericsnowcurrently Oct 15, 2024
a2032a8
Fix a typo.
ericsnowcurrently Oct 15, 2024
54119b8
Fix the documented signature.
ericsnowcurrently Oct 15, 2024
744dca7
Test and document asyncio support.
ericsnowcurrently Oct 15, 2024
f61d62d
Apply suggestions from code review
ericsnowcurrently Oct 16, 2024
ee65bb2
Expand the docs.
ericsnowcurrently Oct 16, 2024
a7f5c50
For now, drop support for scripts.
ericsnowcurrently Oct 16, 2024
b148e09
Fix a TODO comment.
ericsnowcurrently Oct 16, 2024
e365ae7
Fix the docs.
ericsnowcurrently Oct 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Tweak the docs.
  • Loading branch information
ericsnowcurrently committed Oct 8, 2024
commit f8d427304ff0d3651e72c0bf5fb48a1d9d370735
37 changes: 19 additions & 18 deletions Doc/library/concurrent.futures.rst
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ asynchronously executing callables.
The asynchronous execution can be performed with threads, using
:class:`ThreadPoolExecutor` or :class:`InterpreterPoolExecutor`,
or separate processes, using :class:`ProcessPoolExecutor`.
Both implement the same interface, which is defined
Each implements the same interface, which is defined
by the abstract :class:`Executor` class.

.. include:: ../includes/wasm-notavail.rst
Expand Down Expand Up @@ -234,17 +234,17 @@ InterpreterPoolExecutor

The :class:`InterpreterPoolExecutor` class is a :class:`ThreadPoolExecutor`
subclass that uses a pool of isolated interpreters to execute calls
asynchronously. Each interpreter has its own GIL, which allows the
executor to side-step the :term:`Global Interpreter Lock
<global interpreter lock>`, allowing the use of multiple cores.
Interpreters mostly can't share objects between them, which means that,
in most cases, only picklable objects can be executed and returned.
asynchronously. Each interpreter is isolated from the others and thus
can side-step the :term:`Global Interpreter Lock <global interpreter lock>`,
allowing the use of multiple cores. Interpreters mostly can't share
objects between them, which means that, in most cases, only picklable
objects can be executed and returned.

.. class:: InterpreterPoolExecutor(max_workers=None, mp_context=None, initializer=None, initargs=(), shared=None)

A :class:`ThreadPoolExecutor` subclass that executes calls asynchronously
using a pool of at most *max_workers* interpreters. Each interpreter
runs tasks in its own thread.
using a pool of at most *max_workers* threads. Each thread runs
tasks in its own interpreter.

*initializer* and *initargs* are the same as with
:class:`ThreadPoolExecutor`, though they are pickled like with
Expand All @@ -256,25 +256,26 @@ in most cases, only picklable objects can be executed and returned.
Similarly you can pass a script to :meth:`~Executor.submit`, which
will be executed in the interpreter's ``__main__`` module. In that
case no arguments may be provided and the return value is always
``None``.
``None``. Functions (and arguments) are pickled like we do with
the initializer.

For both *initializer* and :meth:`~Executor.submit`, if a script
is passed in then it will automatically have :func:`textwrap.dedent`
applied to it. That means you don't have to.
applied to it. That means you don't have to do so.

:meth:`InterpreterPoolExecutor <Executor.map>` does *not* support
passing in a script.
:meth:`~Executor.map` does *not* support passing in a script.

In each of those cases, an uncaught exception from the initializer
or task is raised as an
:class:`~concurrent.futures.interpreter.ExecutionFailed` exception,
rather than the uncaught exception itself.
or task might not be suitable to send between interpreters, to be
raised as is. In tha case, an
:class:`~concurrent.futures.interpreter.ExecutionFailed` exception
is raised instead which contains a summary of the original exception.

*shared* is an optional dict of objects shared by all interpreters
in the pool. The items are added to each interpreter's ``__main__``
module. Shareable objects include the builtin singletons, :class:`str`
and :class:`bytes`, and :class:`memoryview`. See :pep:`734`
for more info.
module. Not all objects are shareable. Those that are include
the builtin singletons, :class:`str` and :class:`bytes`,
and :class:`memoryview`. See :pep:`734` for more info.

The other caveats that apply to :class:`ThreadPoolExecutor` apply here.
Comment thread
ericsnowcurrently marked this conversation as resolved.
Outdated

Expand Down