-
-
Notifications
You must be signed in to change notification settings - Fork 32k
gh-128041: Add a terminate_workers method to ProcessPoolExecutor #128043
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
36 commits
Select commit
Hold shift + click to select a range
47b162a
gh-128041 - Add a terminate_workers method to ProcessPoolExecutor
csm10495 6ef8833
📜🤖 Added by blurb_it.
blurb-it[bot] 61c9b14
Fix lint
csm10495 3bf5464
Swap to SIGTERM as the default
csm10495 4b285b8
Add some tests
csm10495 b4939fd
Update some docs
csm10495 ba6a4c0
Fix docs
csm10495 5d58e50
Fix docs
csm10495 0db381b
PR fixes/updates
csm10495 7ae1685
SIGKILL doesn't exist on windows
csm10495 f7ad96c
Update Lib/concurrent/futures/process.py
csm10495 2c0b578
Apply suggestions from code review
csm10495 a878221
Fix indenting from suggestions
csm10495 794ee25
Internally call shutdown to prevent a resource leak when calling term…
csm10495 64693a7
Change test to not validate calling of os.kill since shutdown may cal…
csm10495 926dff1
Commit to retrigger CI
csm10495 6d77c10
Merge branch 'python:main' into terminate_workers
csm10495 4429b2f
PR feedback. Split terminate_workers into terminate_workers and kill_…
csm10495 b8d6e5f
Remove un-needed imports
csm10495 f9a7714
lint
csm10495 7cfa42e
Harden a test a bit to ensure the correct type of kill/terminate was …
csm10495 2b31fab
rekick ci
csm10495 ad15ee5
Allow more time for queue to get data back
csm10495 0f57912
Use subTest to break up tests
csm10495 c16fde5
Merge branch 'main' into terminate_workers
csm10495 1bedb28
Apply suggestions from code review
csm10495 f1b0cf6
PR feedback: swap to dict with constants, better subtest parameteriza…
csm10495 cc5f359
swap to using context in the test
csm10495 b3cc8a2
trailing whitespace
csm10495 dbf9d32
Various pr feedbacks
csm10495 1e16da6
PR feedback: swap name of terminate_or_kill to force_shutdown
csm10495 52a5326
PR feedback: swap test names
csm10495 7f09586
PR feedback: use self.executor_type instead of ProcessPoolExecutor di…
csm10495 d5f7578
Add constants for terminate/kill methods
csm10495 0e42eca
feedback to get below 80 chars per pep8
csm10495 3f55347
Merge branch 'main' into terminate_workers
csm10495 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,13 +1,17 @@ | ||
import os | ||
import queue | ||
import signal | ||
import sys | ||
import threading | ||
import time | ||
import unittest | ||
import unittest.mock | ||
from concurrent import futures | ||
from concurrent.futures.process import BrokenProcessPool | ||
|
||
from test import support | ||
from test.support import hashlib_helper | ||
from test.test_importlib.metadata.fixtures import parameterize | ||
csm10495 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
||
from .executor import ExecutorTest, mul | ||
from .util import ( | ||
|
@@ -22,6 +26,19 @@ def __init__(self, mgr): | |
def __del__(self): | ||
self.event.set() | ||
|
||
TERMINATE_WORKERS = futures.ProcessPoolExecutor.terminate_workers.__name__ | ||
KILL_WORKERS = futures.ProcessPoolExecutor.kill_workers.__name__ | ||
FORCE_SHUTDOWN_PARAMS = [ | ||
dict(function_name=TERMINATE_WORKERS), | ||
dict(function_name=KILL_WORKERS), | ||
] | ||
|
||
def _put_sleep_put(queue): | ||
""" Used as part of test_terminate_workers """ | ||
queue.put('started') | ||
time.sleep(2) | ||
queue.put('finished') | ||
|
||
|
||
class ProcessPoolExecutorTest(ExecutorTest): | ||
|
||
|
@@ -218,6 +235,86 @@ def mock_start_new_thread(func, *args, **kwargs): | |
list(executor.map(mul, [(2, 3)] * 10)) | ||
executor.shutdown() | ||
|
||
def test_terminate_workers(self): | ||
mock_fn = unittest.mock.Mock() | ||
with self.executor_type(max_workers=1) as executor: | ||
executor._force_shutdown = mock_fn | ||
executor.terminate_workers() | ||
|
||
mock_fn.assert_called_once_with(operation=futures.process._TERMINATE) | ||
|
||
def test_kill_workers(self): | ||
mock_fn = unittest.mock.Mock() | ||
with self.executor_type(max_workers=1) as executor: | ||
executor._force_shutdown = mock_fn | ||
executor.kill_workers() | ||
|
||
mock_fn.assert_called_once_with(operation=futures.process._KILL) | ||
|
||
def test_force_shutdown_workers_invalid_op(self): | ||
with self.executor_type(max_workers=1) as executor: | ||
self.assertRaises(ValueError, | ||
executor._force_shutdown, | ||
operation='invalid operation'), | ||
|
||
@parameterize(*FORCE_SHUTDOWN_PARAMS) | ||
def test_force_shutdown_workers(self, function_name): | ||
manager = self.get_context().Manager() | ||
q = manager.Queue() | ||
|
||
with self.executor_type(max_workers=1) as executor: | ||
executor.submit(_put_sleep_put, q) | ||
|
||
# We should get started, but not finished since we'll terminate the | ||
# workers just after | ||
self.assertEqual(q.get(timeout=5), 'started') | ||
|
||
worker_process = list(executor._processes.values())[0] | ||
getattr(executor, function_name)() | ||
worker_process.join() | ||
|
||
if function_name == TERMINATE_WORKERS or \ | ||
sys.platform == 'win32': | ||
# On windows, kill and terminate both send SIGTERM | ||
self.assertEqual(worker_process.exitcode, -signal.SIGTERM) | ||
elif function_name == KILL_WORKERS: | ||
self.assertEqual(worker_process.exitcode, -signal.SIGKILL) | ||
else: | ||
self.fail(f"Unknown operation: {function_name}") | ||
|
||
self.assertRaises(queue.Empty, q.get, timeout=1) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I do suspect we may see this come up as occasionally flaky in buildbot or CI systems as it is depend on the timing of the sleeps and kills which really can't be guaranteed on a loaded system. if so, _put_sleep_put can have its sleep increased. lets see how it goes first. |
||
|
||
@parameterize(*FORCE_SHUTDOWN_PARAMS) | ||
def test_force_shutdown_workers_dead_workers(self, function_name): | ||
with self.executor_type(max_workers=1) as executor: | ||
future = executor.submit(os._exit, 1) | ||
self.assertRaises(BrokenProcessPool, future.result) | ||
|
||
# even though the pool is broken, this shouldn't raise | ||
getattr(executor, function_name)() | ||
|
||
@parameterize(*FORCE_SHUTDOWN_PARAMS) | ||
def test_force_shutdown_workers_not_started_yet(self, function_name): | ||
ctx = self.get_context() | ||
with unittest.mock.patch.object(ctx, 'Process') as mock_process: | ||
with self.executor_type(max_workers=1, mp_context=ctx) as executor: | ||
# The worker has not been started yet, terminate/kill_workers | ||
# should basically no-op | ||
getattr(executor, function_name)() | ||
|
||
mock_process.return_value.kill.assert_not_called() | ||
mock_process.return_value.terminate.assert_not_called() | ||
|
||
@parameterize(*FORCE_SHUTDOWN_PARAMS) | ||
def test_force_shutdown_workers_stops_pool(self, function_name): | ||
with self.executor_type(max_workers=1) as executor: | ||
task = executor.submit(time.sleep, 0) | ||
self.assertIsNone(task.result()) | ||
|
||
getattr(executor, function_name)() | ||
|
||
self.assertRaises(RuntimeError, executor.submit, time.sleep, 0) | ||
|
||
|
||
create_executor_tests(globals(), ProcessPoolExecutorTest, | ||
executor_mixins=(ProcessPoolForkMixin, | ||
|
4 changes: 4 additions & 0 deletions
4
Misc/NEWS.d/next/Library/2024-12-17-18-53-21.gh-issue-128041.W96kAr.rst
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,4 @@ | ||
Add :meth:`concurrent.futures.ProcessPoolExecutor.terminate_workers` and | ||
:meth:`concurrent.futures.ProcessPoolExecutor.kill_workers` as | ||
ways to terminate or kill all living worker processes in the given pool. | ||
(Contributed by Charles Machalow in :gh:`128043`.) |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.