Background job systems like Sidekiq use a Sorted Set as a timestamp-sorted priority queue: (epoch_seconds, job) so you'll have elements like:
ZADD retry 1764016057.428448 "{job payload}"
meaning that the job should retry at 1764016057.428448. In other words, the timestamp indicates WHEN the element should be processed, somehow.
Today we have to use a Lua script which combines ZRANGE and ZREM and constantly poll it to find if any elements have passed the current time. I'd like to see a blocking command which treats the score as a timestamp and blocks until the head has passed the current time so we don't need this polling. This would:
- dramatically reduce unnecessary polling.
- provide much more accurate job scheduling since accuracy and efficient polling are mutually exclusive.
I'd propose possible new commands BZTSPOP (sorted set blocking timestamp pop) and BZTSMOVE (sorted set blocking timestamp pop):
- BZTSPOP [sortedset] [up_to_count] [timeout]
- BZTSMOVE [sortedset] [list] [up_to_count] [timeout]
The former pops multiple values as long as they are all past time. The latter pushes value(s) to the given named list, which provides the data safety and reliability that POP does not.
If you have alternative suggestions for how to remove/reduce Redis polling, I would be very happy to discuss.