Summary

This is a request for implementing a ratelimit layer with sliding window strategy.

Current status

The current RateLimit implemention appears to be fixed window. I.e. you can think of it like this:

1. Start a clock
2. Count requests while keeping an eye on the quota
3. Throttle requests if the quota is up
4. Reset the quota when the times up
5. Repeat

(This could be documented more explicitly too. I initially believed that it would be sliding window.)

Sliding window

For quota Q and duration T, a sliding window limit guarantees that there hasn't been more than Q calls during the span [now-T, now] (the "window").

It can be thought to work like this: Keep a record of all calls for duration T. When a call comes in, it will be throttled if there's already Q calls within the window. After a while the window will have moved past those previous calls, and thus more quota will be released.

Benefits

• Tends towards less bursty traffic shape than fixed window
• Satisfies all fixed window limits of the same length and quota.
  • This makes it really easy to comply with the common fixed window web API ratelimits. The client doesn't need to worry about syncing its windows with the server.

Cons

• More complex than fixed window limit
• Probably a little more overhead