I have a send-message job that sends a text message. We have a global 3,000 messages / minute rate limit, a limit of 6 messages / second per sending phone number, and have many different clients that send messages. Each client has hundreds of phone numbers.

Our goals are:
a) To stay compliant with our global rate limit
b) To stay compliant with our per-phone number rate limit
c) To prevent any client from clogging the queue for all other clients, such that one client sending 6,000 messages in a minute means that all other clients messages are delayed by 2. Something like 1000 / minute would probably be sensible here, given that not all clients are going to send their maximum volume at once.

One way to do this is a Postgres friendly simplification of the leaky bucket algorithm, where you would have buckets:

buckets (
    bucket_type text,
    interval text, // 'second' or 'minute'
    capacity integer
)

And bucket_intervals:

bucket_intervals (
    bucket_type text,
    bucket_name text,
    bucket_time timestamp, // truncated to the second or minute based on the bucket interval
    tokens_used integer, // number of jobs in this bucket
    primary key (bucket_time, bucket_type, bucket_name)
)

Whenever run_at is computed, if the job has specified buckets (via a buckets text[] column / parameter), run_at would be set to the max of the user specified run_at (or now) and the next available slot which doesn't overflow any bucket interval.

For our use case, our buckets table would be:

insert into buckets (bucket_type, interval, capacity)
values 
    ('send-message-global', 'minute', 3000),
    ('send-message-client', 'minute', 1000),
    ('send-message-phone', 'second', 6);

And each send-message job would be queued with three buckets:

1. send-message-global (which would map to bucket_type = 'send-message-global', bucket_name = null)
2. send-message-client|<client-id> (bucket_type = 'send-message-client', bucket_name = <client-id>)
3. send-message-phone|<phone-number> (bucket_type = 'send-message'phone, bucket_name = <phone-number`)

I think this could be accomplished via:

1. adding an optional buckets parameter to add_job
2. triggers on the jobs table that only run when the job has buckets, and as a result have no negative performance impact for users who don't need this feature

To keep it performant, we would need to delete past buckets. This could either be done on job completion / failure, or we could just write a SQL function to do it and leave periodically calling that function up to the user.

Although I'd like to be able to use an approach similar to the one described here, in this case we have multiple different queues whose rate limits interact.

Although it's also possible to implement this in user land with a custom add_job function and by overwriting fail_job, the project it is part of is closed source, and other users may benefit from having a simple way to rate limit jobs that interact with rate limited external APIs.

Do you think this is a good solution / would you like a PR for it, or do you think this level of complexity is best kept outside of this project?