Thanks to visit codestin.com
Credit goes to github.com

Skip to content

gh-96471: Add asyncio queue shutdown #104228

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
gvanrossum merged 28 commits into from
Apr 6, 2024
Merged

gh-96471: Add asyncio queue shutdown #104228

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
440a702
Add asyncio queue shutdown
EpicWink Sep 1, 2022
fb458db
Fix queue shutdown
YvesDup Feb 10, 2023
e5951ac
📜🤖 Added by blurb_it.
blurb-it[bot] May 6, 2023
a72aedd
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Feb 20, 2024
d5e925d
Add references in docs and news entry
EpicWink Feb 20, 2024
f3517fb
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 20, 2024
bd2a7c3
Improve docs
EpicWink Mar 20, 2024
e9ac8de
Consume queue on immediate shutdown
EpicWink Mar 20, 2024
1e7813a
Fix links in what's-new
EpicWink Mar 22, 2024
1275bb6
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 22, 2024
eec29bb
Fix formatting in news entry
EpicWink Mar 22, 2024
2c6156f
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 22, 2024
17f1f32
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 26, 2024
a233830
Improve tests
EpicWink Mar 26, 2024
420a247
Improve tests even more
EpicWink Mar 26, 2024
25ad2ac
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 26, 2024
f3321b4
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 27, 2024
6d9edd6
Document tests
EpicWink Mar 27, 2024
1135d85
Merge remote-tracking branch 'upstream/main' into asyncio-queue-shutdown
EpicWink Mar 28, 2024
ddc6ad6
Always allow getters to re-check queue empty
EpicWink Mar 28, 2024
2fa1bd9
Merge branch 'main' into asyncio-queue-shutdown
gvanrossum Apr 3, 2024
aef4063
Simplify shutdown-check in put and get
EpicWink Apr 4, 2024
d49c6dd
Format shutdown docstring
EpicWink Apr 4, 2024
5a435a6
Check for 0 unfinised tasks in shutdown
EpicWink Apr 4, 2024
c8db40e
Use asyncio.sleep to run other tasks
EpicWink Apr 4, 2024
ca01ee1
Use public method to shut down queue in format test
EpicWink Apr 4, 2024
b02c4dd
Only start queue join after shutdown in test
EpicWink Apr 4, 2024
8deca77
Test join before failing task-done
EpicWink Apr 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add references in docs and news entry
  • Loading branch information
@EpicWink
EpicWink committed Feb 20, 2024
commit d5e925d7be1140c2db34d003a22d4afaa748bce9
14 changes: 8 additions & 6 deletions Doc/library/asyncio-queue.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -102,14 +102,16 @@ Queue

.. method:: shutdown(immediate=False)

Shut-down the queue, making queue gets and puts raise
:exc:`QueueShutDown`.
Shut down the queue, making :meth:`~Queue.get` and :meth:`~Queue.put`
raise :exc:`QueueShutDown`.

By default, gets will only raise once the queue is empty. Set
*immediate* to true to make gets raise immediately instead.
By default, :meth:`~Queue.get` on a shut down queue will only raise once
the queue is empty. Set *immediate* to true to make gets raise
Comment thread
EpicWink marked this conversation as resolved.
Outdated
immediately instead.

All blocked callers of put() will be unblocked, and also get()
and join() if *immediate* is true.
All blocked callers of :meth:`~Queue.put` will be unblocked. If
*immediate* is true, also unblock callers of :meth:`~Queue.get` and
:meth:`~Queue.join`.

Copy link
Copy Markdown
Contributor

@YvesDup YvesDup Apr 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry but I have a doubt, shouldn't this documentation block be rather:

    All blocked callers of :meth:`~Queue.put` and :meth:`~Queue.get` 
    will be unblocked. If *immediate* is true, also unblock callers of 
    :meth:`~Queue.join`.

In event of change, the docstring of the shutdown method must be updated.

Copy link
Copy Markdown
Contributor Author

@EpicWink EpicWink Apr 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

join callers aren't necessarily even unblocked anyway, if consumers are processing any items. I should probably say that a task is marked as done for each item in the queue if immediate shutdown.

Also, I think the threading queue docs are the same.

Copy link
Copy Markdown
Contributor

@YvesDup YvesDup Apr 7, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a task is marked as done for each item in the queue if immediate shutdown.

It's very precise, better.

Also, I think the threading queue docs are the same.

Yes, I commented here so as not to forget (see #117532 (comment)).

English is your native language, I think it'is best if you update documentations and docstrings.

Update: but I can create the follow-up PR.

Copy link
Copy Markdown
Contributor Author

@EpicWink EpicWink Apr 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've made a PR: #117621

.. versionadded:: 3.13

Expand Down
3 changes: 2 additions & 1 deletion Misc/NEWS.d/next/Library/2023-05-06-05-00-42.gh-issue-96471.S3X5I-.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
Add asyncio.Queue termination with ``shutdown`` method.
Add :py:class:`asyncio.Queue`` termination with
:py:meth:`~asyncio.Queue.shutdown` method.