Fixes #7052 gevent/eventlet 1 seconds latency#7089
Merged
auvipy merged 2 commits intocelery:masterfrom Nov 15, 2021
Merged
Conversation
Wakeup waiters in `wait_for` after every `drain_events` occurs instead of only after 1 seconds timeout. Does not block event loop, because `drain_events` of asynchronous backends with pubsub commonly sleeping for some nonzero time while waiting events.
for more information, see https://pre-commit.ci
Codecov Report
@@ Coverage Diff @@
## master #7089 +/- ##
==========================================
+ Coverage 89.35% 89.37% +0.01%
==========================================
Files 138 138
Lines 16744 16759 +15
Branches 2212 2211 -1
==========================================
+ Hits 14962 14978 +16
- Misses 1551 1552 +1
+ Partials 231 229 -2
Flags with carried forward coverage won't be shown. Click here to find out more.
Continue to review full report at Codecov.
|
|
This pull request introduces 1 alert and fixes 2 when merging 366258e into fe37cd8 - view on LGTM.com new alerts:
fixed alerts:
|
Member
|
thank you, will look into this tomorrow |
auvipy
approved these changes
Nov 15, 2021
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue #7052
When there are used asynchronous backend as Redis with pubsub, result of short task (0.1s length, for example) available immediate.
When execute AsyncResult.get() under gevent/eventlet environment, result of the same short (0.1s) tasks became available after 1s, it is too slowely.
Current behavior is result of GH-5974 that prevent event loop blocking by very offen checking task results and very short wait_for length.
Current implementation of greenletDrainer cause wait_for to only return every "timeout" # of seconds, rather than returning immediately.
Solutuion:
I add new internal event
_drain_complete_eventthat rising every time after socket operation result_consumer.drain_events (for example _pubsub.get_message for Redis backend) occurs.Every
wait_forare waiting for this particular event, so it won't cause overhead, because socket operation still not block event loop and let other greenlets executing normaly.