Unsure of sane defaults

[lcpriest]

This might be a stupid question.

Is there a recommended default_worker_options for sidekiq-unique-jobs?

I've been testing a few different strategies and I'm still not sure what is the best strategy.

I have a bunch of jobs that should not be run at the same time, but can be run one after the other as there is some idempotency built-in. I have other jobs that run every day (fetching data from APIs), that have the exact same args.

I'm currenly using while_executing, but it seems to lock up my queues a lot.

Sidekiq.default_worker_options = {
  lock: :while_executing,
  lock_timeout: nil

Any recommendations would be appreciated - if this is an inappropriate questions, happy to have it insta-closed.

[mhenrixon]

Great question @lcpriest ! This question just made me realize I totally approached the whole README wrong. Examples like you requested would have been a better README than what it currently is.

It is so individual, for most jobs I’d say while executing is the most appropriate but unfortunately it behaves seemingly weird. Since the lock is handled as the server is processing the job and not when the client pushes it. Many server processes may seem to process the same job when in fact the subsequent processors are just waiting for the first one to finish. Avoiding a huge lock timeout and using conflict resolution strategies becomes imperative but complicated.

UntilExecuted is probably the most reliable. It locks from when it is pushed by the client and unlocks when it is done being processed by the sidekiq server processor. It does prevent all duplicates from even being queued but it is super obvious what is going on.

I can’t really make an absolute recommendation. The gem can never replace idempotency using a single point of truth like a RDBMS on your side.

This gem is mostly a supplement to avoid that extra database/api call. Currently the gem does almost too good of a job at preventing duplicates.

_{Sent with GitHawk}

[lcpriest]

I really appreciate the in-depth response!

This question just made me realize I totally approached the whole README wrong.
Totally reasonable, more often than not you get tickets from people with edge cases, so you will write a README that gives as much detail as possible.

I have currently settled on until_executed, though I think I will need to regularly wipe out the unique keys as I have had instances of keys remaining in Redis after the job has long finished. I assume this is partially to do with the deploy process on Heroku causing Sidekiq to be a bit funky on the shutdown and startup.

Thanks again and thank you for the work you do. It is really appreciated!