Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Add deterministic alternatives for asyncio.wait and asyncio.as_completed #533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jun 4, 2024

Conversation

cretz
Copy link
Member

@cretz cretz commented May 28, 2024

What was changed

asyncio.as_completed and asyncio.wait are non-deterministic sometimes so we need alternatives.

Checklist

  1. Closes [Bug] asyncio.wait is non-deterministic when used with coroutines instead of tasks #429
  2. Closes [Feature Request] Investigate asyncio.as_completed and asyncio.wait usage of set as non-deterministic #518

@cretz cretz requested a review from a team as a code owner May 28, 2024 14:55
@cretz cretz force-pushed the asyncio-non-det branch from 87e80e4 to 2bc0718 Compare May 28, 2024 15:03
deterministic:

* `asyncio.as_completed()` - use `workflow.as_completed()`
* `asyncio.wait()` - use `workflow.wait()`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering about naming these workflow.asyncio_wait() andworkflow.asyncio_as_completed().

The main reason is that workflow.wait() will be confused with workflow.wait_condition(). It may just be a hiccup that they'll get over, but it would be nice not to cause users that hiccup.

Another reason is that workflow.asyncio_xxx() is just directly saying exactly what it is: a shim that is basically exactly the same as the utility from asyncio. It makes it clear that they are utilities for dealing with arbitrary tasks/coroutines rather than being a special sort of as_completed or wait that has to do with waiting or completing Temporal-specific entities.

Copy link
Member Author

@cretz cretz May 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered it, but we don't have asyncio_sleep, just sleep. I think these don't need to necessarily be seen as asyncio utilities, they can be seen as workflow utilities that just so happen to be in asyncio as well. I think they work well as workflow utilities just like wait_condition. (same for uuid4() and time() and random() and other Python copies we don't prefix with their originating module name + _)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered it, but we don't have asyncio_sleep, just sleep

I don't really see the analogy there: we don't have workflow.sleep(), instead users just use asyncio.sleep(). Here, users would call asyncio.xxx() but they shouldn't, so we point them to workflow.asyncio_xxx().

workflow.wait_condition() is an extremely important API and I really worry that introducing workflow.wait() will confuse users and damage DX. It's a very similar name to workflow.Await() / Workflow.await() that users may have come across in the Go and Java SDKs, and IDEs will offer it first when users are exploring the workflow namespace. I'd like to wait for other team opinions on this.

Copy link
Member Author

@cretz cretz May 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't really see the analogy there

Sorry, correct, I was thinking of other SDKs that do have sleep and don't prefix it. In Python we are able to reuse sleep. We do have time(), time_ns(), random(), and uuid4() though that are copies of other Python module functions and we don't prefix those.

workflow.wait_condition() is an extremely important API and I really worry that introducing workflow.wait() will confuse users and damage DX

Python has wait and wait_for which are different things too and if they can expect users to understand the difference I think we can too.

I'd like to wait for other team opinions on this

👍

@cretz
Copy link
Member Author

cretz commented May 29, 2024

After team discussion, it was decided to issue warnings if the asyncio versions are called. The PR has been updated to support this. Please re-review.

@cretz cretz requested a review from a team May 29, 2024 22:26
@cretz cretz merged commit 365cead into temporalio:main Jun 4, 2024
12 checks passed
@cretz cretz deleted the asyncio-non-det branch June 4, 2024 14:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants