Add burst control to the existing Stream throttling combinators #1205

vasilmkd · 2019-07-19T18:56:40Z

Also fix potential token overflow.

Should be enough to close #1027.

iravid · 2019-07-19T19:43:49Z

@vasilmkd would it be possible to add burst as a default parameter on all the variants instead of a separate variant?

vasilmkd · 2019-07-19T23:37:27Z

The current implementation of the token bucket algorithm should now closely match the one described here: Token Bucket Wikipedia.

iravid · 2019-07-20T19:44:59Z

streams/shared/src/main/scala/zio/stream/ZSink.scala

-      _       <- assertPositive(units).toManaged_
-      current <- clock.currentTime(TimeUnit.NANOSECONDS).toManaged_
-      bucket  <- Ref.make((units, current)).toManaged_
+    def checkTokens(sum: Long, max: Long): Long = if (sum < 0) max else math.min(sum, max)


Question: when sum < 0, should we subtract it from max? Otherwise when the bucket is depleted, there is always constant amount of max available in addition.

You're right, I'll fix this.

Actually, on second thought, the checkTokens function is called to guard against overflow. sum is computed exclusively from positive values, whose total value might exceed Long.MaxValue (which is what the if statement checks). If that happens, we want max tokens to be available, otherwise, if we are dealing with positive values (sum and max), just get the minimum value.

iravid · 2019-07-20T19:46:01Z

Also, I believe the default value for max should reflect no burst capacity, so Long.MaxValue is probably not a good choice for that.

vasilmkd · 2019-07-20T19:59:44Z

There are 2 ways to do this.

Use max as the maximum number of tokens allowed at any time in the bucket (currently implemented). At the moment, the default value of Long.MaxValue enables an average bandwidth of units / duration. In order to restrict burst to 0, and offer at best units / duration bandwidth, one needs to set max to be equal to units. I personally think that having the maximum number of tokens be configurable is more user friendly than the second option. However, the no burst default needs to be implemented as an overloaded method, since max cannot be initialized to units in the type signature.
Use threshold as a number of tokens which can be added to units when burst occurs. In the no burst default, it would be an argument with default value 0. This approach however, increases the danger of overflow, as two values of type Long need to be added together.

Interested to hear your thoughts on this. I'm happy to implement any of them.

vasilmkd · 2019-07-20T20:28:05Z

Idea.

I can implement the second proposal from the last comment by wrapping the existing functions in
throttleEnforceM(units: Long, duration: Duration, burst: Long = 0): ZManaged[ZSink] which simply delegates to the inner method throttleEnforceM(units, duration, units + burst or Long.MaxValue in case of overflow) (the current implementaion).

That way we get the semantics of the second proposal with the "overflow safety" of the first one, and as a bonus, it requires minimal code changes.

…ttle

iravid · 2019-07-21T18:42:12Z

Hi @vasilmkd, could you explain why you say that sum is computed exclusively of positive values?

I infer from what you say that the tokens in the ref could never be negative. However:

remaining is computed as available - weight;
if available < weight, then remaining is negative
a negative number is stored in the ref
at the next element, assuming not enough cycles have passed, tokens + (cycles * units) is still negative, so maxTokens is used
assuming that element is heavy enough, we still go into negative values for remaining;
the next round repeats the same, effectively not throttling.

WDYT?

vasilmkd · 2019-07-21T18:50:10Z

Yes, tokens can become negative. However, the sink is sleeping for enough time required such that tokens amount of tokens can be accumulated, thus at the time when the next element is processed, there are at least 0 tokens.

iravid · 2019-07-21T18:51:08Z

Oh that's true! Thank you for pointing that out. So I'll read your comments about the burst implementation again and reply.

vasilmkd · 2019-07-21T18:52:33Z

No problem. I was reading through the source code of akka-streams and the implementation of this is quite similar. It seems that they don't care about clock precision as much though.

iravid · 2019-07-21T19:23:07Z

@vasilmkd The current implementation (and the units + burst trick) looks good to me! It's a very elegant solution.

This is good to merge by me, unless there's anything else you wanted to address.

vasilmkd · 2019-07-21T19:24:35Z

Happy with how everything works now.

iravid · 2019-07-21T19:26:58Z

Awesome work, thank you @vasilmkd!

…1205) * Change managed sink environment and error type * Add burst control to throttleEnforceM * Test ZSink.throttleEnforce with no burst * Add burst control to ZSink.throttleShape * Change the burst control API and default arguments for enforcing throttle * Change burst control API and default argument value for shape throttle * Tweak ZSink.throttleShape test to show lack of burst * Add ZSink.throttleShape test with burst

vasilmkd changed the title ~~Add ZStream#throttleEnforceBurst, ZStream#throttleEnforceBurstM~~ Add burst control to the existing Stream throttling combinators Jul 20, 2019

iravid reviewed Jul 20, 2019

View reviewed changes

vasilmkd added 8 commits July 21, 2019 11:15

Change managed sink environment and error type

19ac8bc

Add burst control to throttleEnforceM

be6f3e6

Test ZSink.throttleEnforce with no burst

b1a6b82

Add burst control to ZSink.throttleShape

7bfbaae

Change the burst control API and default arguments for enforcing thro…

ac9d464

…ttle

Change burst control API and default argument value for shape throttle

aff0806

Tweak ZSink.throttleShape test to show lack of burst

0062143

Add ZSink.throttleShape test with burst

6585b70

iravid approved these changes Jul 21, 2019

View reviewed changes

iravid merged commit 58c4081 into zio:master Jul 21, 2019

vasilmkd deleted the feature/throttle-burst branch July 21, 2019 19:40

Uh oh!

Add burst control to the existing Stream throttling combinators #1205

Add burst control to the existing Stream throttling combinators #1205

Uh oh!

Conversation

vasilmkd commented Jul 19, 2019

Uh oh!

iravid commented Jul 19, 2019

Uh oh!

vasilmkd commented Jul 19, 2019

Uh oh!

iravid Jul 20, 2019

Choose a reason for hiding this comment

Uh oh!

vasilmkd Jul 20, 2019

Choose a reason for hiding this comment

Uh oh!

vasilmkd Jul 20, 2019

Choose a reason for hiding this comment

Uh oh!

iravid commented Jul 20, 2019

Uh oh!

vasilmkd commented Jul 20, 2019

Uh oh!

vasilmkd commented Jul 20, 2019

Uh oh!

iravid commented Jul 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vasilmkd commented Jul 21, 2019

Uh oh!

iravid commented Jul 21, 2019

Uh oh!

vasilmkd commented Jul 21, 2019

Uh oh!

iravid commented Jul 21, 2019

Uh oh!

vasilmkd commented Jul 21, 2019

Uh oh!

iravid commented Jul 21, 2019

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

iravid commented Jul 21, 2019 •

edited

Loading