-
-
Notifications
You must be signed in to change notification settings - Fork 31.9k
asyncio.TaskGroup may not cancel all tasks on failure of one #94398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
My gut reaction is we disallow new tasks in that Task group; it's shutting down and closed for business. If a task inside wants to spawn new tasks during cleanup it can create a new TG then and there, right? |
That would be an easy fix (just two lines in |
Once the task group is shutting down, it should not be possible to create a new task. Here "shutting down" means `self._aborting` is set, indicating that at least one task has failed and we have cancelled all others.
A sample that reproduces this problem would indeed be welcome. |
The new test I added to the PR might help: In that test, we could extend |
Once the task group is shutting down, it should not be possible to create a new task. Here "shutting down" means `self._aborting` is set, indicating that at least one task has failed and we have cancelled all others. Co-authored-by: Łukasz Langa <[email protected]>
…GH-94400) Once the task group is shutting down, it should not be possible to create a new task. Here "shutting down" means `self._aborting` is set, indicating that at least one task has failed and we have cancelled all others. Co-authored-by: Łukasz Langa <[email protected]> (cherry picked from commit 594c369) Co-authored-by: Guido van Rossum <[email protected]>
…H-94463) Once the task group is shutting down, it should not be possible to create a new task. Here "shutting down" means `self._aborting` is set, indicating that at least one task has failed and we have cancelled all others. Co-authored-by: Łukasz Langa <[email protected]> (cherry picked from commit 594c369) Co-authored-by: Guido van Rossum <[email protected]>
While writing documentation for
asyncio.TaskGroup
I discovered something fishy.There is a behavior where if a task fails with an exception (except
asyncio.CancelledError
), the remaining tasks are cancelled. There is also a rule that this is only done once, implemented usingself._aborted
, which is set byself._abort()
.But what should happen if new tasks are created after
self._abort()
is called, and one of those tasks fails? Then the remaining new tasks are not cancelled. To repro, we need something that creates two tasks, where the first one fails, and the second catchesasyncio.CancelledError
and when caught creates two more tasks. The third task would then fail, and the fourth task might wait for ever, never getting cancelled.Is this a bug (or design flaw)? I think we decided we would support task creation during the wait (even though EdgeDb's TaskGroup disallowed it) so that it's possible to dynamically create new tasks forever -- this seems useful. But I'm not sure we thought deep about whether to allow creating new tasks once we're waiting for all cancelled tasks to finish.
How would we fix it? Disallowing new task creation once
self._aborted
is set seems excessive, since it would disallow legitimate creation of new tasks during cleanup. We could keep a weak set of tasks that we haven't cancelled yet, and if one of those fails we could cancel all others in that set (and remove them from the set) -- this would essentially create successive "generations" of tasks that live or die together (starting a new generation once any member of the current generation dies). Is this worth it?CC: @njs @Tinche @agronholm -- I assume this problem doesn't exist in Trio because of its level-triggered cancellation, but maybe one of you still has a useful insight.
The text was updated successfully, but these errors were encountered: