I just discovered that jobs are unlocked if they fail. This happens regardless if there are retries left or if the job dies. I'm wondering if this is the intended behavior. As I would understand (and need) that fact a job that will be retired should still be unique.
The code responsible for this is the following: (in /lib/sidekiq_unique_jobs/middleware/server/unique_jobs.rb)
def call(worker, item, _queue, redis_pool = nil)
...
yield
ensure
if after_yield? || !defined? unlocked || unlocked != 1
unlock(lock_key)
end
endSo are there any reasons for this behavior or do I miss anything?
Update
So after some research I start to understand why unlocking works the way it does. I was not aware of the fact that jobs in the schedule/retry queue are pushed to the worker queues via client push (involving sidekiqs client middlewares). If in that situation the job is still locked the job will never be pushed into its worker queue.
To circumvent this issue it might be possible to save the jid of the job that acquired the lock instead of the [1,2]. So the client middleware can check for the jid and let the job reenqueue if they match.
What do you think of this idea?
I just discovered that jobs are unlocked if they fail. This happens regardless if there are retries left or if the job dies. I'm wondering if this is the intended behavior. As I would understand (and need) that fact a job that will be retired should still be unique.
The code responsible for this is the following: (in /lib/sidekiq_unique_jobs/middleware/server/unique_jobs.rb)
So are there any reasons for this behavior or do I miss anything?
Update
So after some research I start to understand why unlocking works the way it does. I was not aware of the fact that jobs in the schedule/retry queue are pushed to the worker queues via client push (involving sidekiqs client middlewares). If in that situation the job is still locked the job will never be pushed into its worker queue.
To circumvent this issue it might be possible to save the jid of the job that acquired the lock instead of the [1,2]. So the client middleware can check for the jid and let the job reenqueue if they match.
What do you think of this idea?