-
Notifications
You must be signed in to change notification settings - Fork 97
Custom slot supplier #690
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom slot supplier #690
Conversation
4889426
to
aaa7e45
Compare
aaa7e45
to
d7002ed
Compare
5f9e9b7
to
5639b05
Compare
My only remaining concern is I will sometimes see lines like this on test shutdown:
I also occasionally will see them for stuff I didn't touch, though. AFAICT this is simply whining that a cancelled task didn't get one more go in the event loop to have the |
temporalio/worker/_tuning.py
Outdated
@@ -6,9 +8,22 @@ | |||
from typing_extensions import TypeAlias | |||
|
|||
import temporalio.bridge.worker | |||
from temporalio.bridge.worker import ( | |||
ActivitySlotInfo, | |||
CustomSlotSupplier, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we define this and all its other public API outside of the bridge area? Here in this file should be fine. Note we did similar for things like temporalio.runtime.BufferedMetric
as a "protocol" (i.e. interface) noting that it had to match the Rust form of a bridge class impl. If there are things we need to work out with imports we can.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah it's kind of pain because of circular import issues. I'll see if I can move it but I'm worried it means duping a bunch of stuff
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done
temporalio/bridge/src/runtime.rs
Outdated
tokio::runtime::Builder::new_multi_thread(), | ||
TokioRuntimeBuilder { | ||
inner: tokio::runtime::Builder::new_multi_thread(), | ||
lang_on_thread_start: Some(move || { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you help me understand this a bit? Looking at https://github.com/awestlake87/pyo3-asyncio/blob/master/src/tokio.rs I don't see where they are building a Tokio runtime w/ custom thread starter to add task locals for every thread. I wonder if we can just make/set the task locals with the event loop inside the one place they are needed later? The current event loop is available there too. I also see they use https://docs.rs/tokio/latest/tokio/macro.task_local.html, unsure if we could too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
They don't because I think it's built mostly on the assumption that it's Python calling into Rust and not the other way around, so any time and event loop is needed it already exists. That's not the case here, because we call into Python from whatever arbitrary thread in the Tokio runtime and that thread may not have any loop available on it, so we need to re-use the existing one (which also needs to be the same as the main one to avoid surprise issues with different loops trying to use things that were created on the main/another loop).
loop inside the one place they are needed later
What one place are you thinking of? Task locals are I think just going to mean we call this way more frequently than thread locals.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What one place are you thinking of?
Ah, yes, my mistake. So question, can any of these Python references information be captured and moved into the CustomSlotSupplierOfType
? I admit I haven't dug into this, but I know I found in Ruby a way to keep "opaque" references that needed the GIL to use them, but were still Send/Sync so that I could move around (and they were prevented from being collected by Python while still having a reference in Rust).
If that's possible, I think we should put the TaskLocals
on the CustomSlotSupplierOfType
. Granted that assumes it's created with an event loop intact, but that should be an ok assumption I think. Actually, that's not something we should assume with Runtime
now that I think about it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, might be able to do that. I'll take a look
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done. Honestly not sure it's any better, really, but done regardless
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, though it is nice that it's worker specific and not actually setting any task locals
934de2c
to
6ca80a5
Compare
4015670
to
9f0f51d
Compare
5b95612
to
ec96168
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nothing blocking
temporalio/bridge/src/runtime.rs
Outdated
tokio::runtime::Builder::new_multi_thread(), | ||
TokioRuntimeBuilder { | ||
inner: tokio::runtime::Builder::new_multi_thread(), | ||
lang_on_thread_start: Some(move || { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
True, though it is nice that it's worker specific and not actually setting any task locals
} | ||
} | ||
|
||
fn available_slots(&self) -> Option<usize> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I assume there is no value in allowing users to define this, at least not at this time.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, there's not at the moment
except asyncio.CancelledError: | ||
raise | ||
except Exception: | ||
logger.warning( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you'd rather, instead of logging and having a global logger, can call https://docs.python.org/3/library/warnings.html#warnings.warn instead. We do this in other places for should-not-happen things without relying on them setting up logging.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Decided not to do this since formatting for the user thrown exception is not nearly as nice w/ the stack trace not being shown whereas logging does
What was changed
Add user-implementable slot suppliers
Why?
Parity. Cool feature.
Checklist
Closes
How was this tested:
Added tests
Any docs updates needed?
Yes, docs to follow when .NET is done