Add ZStream#throttle (#1027) #1155
Conversation
|
Hey @vasilmkd, this is awesome! Thank you for this contribution, this is going to be super useful. I'll get to work on reviewing this tomorrow. |
There was a problem hiding this comment.
@vasilmkd This is great stuff, and written very clearly. Nice work!
I have a couple of general comments:
- Can we make the signature for the sink and stream more "user friendly"? E.g. something like:
// Allows a rate of units/duration; e.g. throttle(1024, 1.minute)((a: Byte) => ZIO.succeed(1)) will
// allow for 1Kb/minute
def throttle(units: Long, duration: Duration)(costFn: A => ZIO[R1, E1, Long])- Do we actually need the refill to run in a fiber? You could probably query the current time on each element, compute the time passed since the last element and derive the amount of tokens that need to be refilled. This can make the sink simpler and easier to test.
- This is an enforcing throttle, right? That is - we're just dropping elements that cross the allowed weight. I did not consider this usecase but it's pretty important! Another usecase that we need is a shaping throttle: we allow elements to pass through with the delay required for the bucket to be replenished with enough tokens for them to pass.
- I'd love for this to be driven by a schedule but schedule might be too general for this problem. I'll spend some more time thinking how it can be done.
- We need to account for burst as well. This can be implemented by allowing the bucket to go into negative up to an allowed threshold.
Of these comments, only the first 2 need to be addressed in this PR. The rest can be done in a follow-up.
Thanks again for your work on this!
|
Awesome suggestions. I feel like I've overengineered the solution a bit. Thanks for the insightful comments. |
|
I'm interested to know if it's possible to integrate the test environment under testkit/ (TestClock in particular for this issue). It would be helpful in order to write a deterministic throttling test. However, I think it should be done in a different PR, as it will probably involve some build file manipulation. |
|
@vasilmkd Would be great if you could create follow-up tickets for things that should be deferred :-) |
…io#1155) * Implement throttle ZSink * Implement throttle ZStream * Test throttle ZStream * Change remainder type from A to Nothing * Implement throttle without environment * Redesign throttle without concurrent fiber * Simplify Sink throttle API * Simplify Stream throttle API * Test redesigned Stream throttle * Fix refill timestamp arithmetic * Fix ZSink stream initialization order * Add deterministic ZSink throttle test * Rename effectful throttle to throttleEnforceM * Introduce throttleEnforce pure alias
Token bucket throttling implementation.
I would have liked to use ZSchedule to control the bucket refilling, but I can't seem to figure out how to create a schedule which adds an initial delay to the refilling effect, otherwise, the bucket is initialized with the configurable initial capacity and immediately refilled without any delay. Any help is appreciated.
This is my first open-source contribution. Looking forward to the comments.