-
Notifications
You must be signed in to change notification settings - Fork 2
Look into using Advisory Locks #3
Description
Description
I was linked to: Que a job queue in ruby that claims superior performance by using Advisory locks. This is true advisory locks are faster, and would require some work (since you're essentially implementing you're own concurrency). However, there's several questions left to answer:
- What happens to advisory locks in a failover scenario? It seems like they're entirely lost (since they are stored in a shared memory region, and not flushed to disk). We could use a notification here, but notifications aren't completely durable. I still see a row update needing to happen otherwise (which negates the benefits).
- Advisory locks seem to use the same shared memory region that PostgreSQL uses for connections. To quote the docs:
Care must be taken not to exhaust this memory or the server will be unable to grant any locks at all. This imposes an upper limit on the number of advisory locks grantable by the server, typically in the tens to hundreds of thousands depending on how the server is configured.
Between the number of connections, and locks for a potentially large job queue this could be really catastrophic. As the number of jobs could create so many locks normal behavior that uses locks can't go through.
Now none of these are necissarily deal breakers (perhaps we can offer an "unsafe" mode of execution that uses locks, for the extra performance gain; where we can then document the pitfalls). That being said research needs to be done in order to determine, what the best way to handle these scenarios are.