Sidekiq in Practice

July 2022

Reading time: 2 mins

Article Table of Contents

Companion notes in a different repo for that course: https://github.com/josh-works/sidekiq_in_practice/tree/main

Similar to what I did for Noah Gibb’s Rebuilding Rails Course

draft notes, scratch pad, etc

Context: I joined a team at Homebot that deals with lots of background jobs, via Sidekiq and Redis. lots of jobs, every day. Many queueus. Time matters. Performance matters. So, here we go, deep dives, left and right.

I’m using this to store notes/code snippets for me, and others.

Tips on Sidekiq Queues (Phil.tech) #

I had to reference https://josh.works/sidekiq-and-background-jobs-in-rails-for-beginners#making-sidekiq-do-stuff-via-the-rails-console a few times, when I borked job enquing, and they all blew up hard.

Working through the above resource, for starters:

#  config/sidekiq-importance.yml
:concurrency: 5
:queues:
  - foo
  - bar
  - baz
  - nevergonnagiveyouup

# app/jobs/priority_test_job.rb
class PriorityTest
  include Sidekiq::Worker

  def perform(*args)
    Rails.logger.info args
    Rails.logger.info "prioritytest: current time: #{Time.now}"
    Rails.logger.info 3 + 3 # difficult & computationally expensive operation
    sleep 3
  end
end

start redis, then sidekiq:

redis-server
bundle exec sidekiq -C ./config/sidekiq-importance.yml

in console:

require 'sidekiq/api'
Sidekiq.strict_args!(false)
Sidekiq::Extensions.enable_delay!

queues = [:foo, :bar, :bazzzzz, :nevergonnagiveyouup]

50.times do |i|
  queues.each do |level|
    PriorityTest.perform_async(queue: level)
    p "#{i}: #{level}"
  end
end

If jobs are stuck, might need:

require 'sidekiq/api'
Sidekiq::Queue.all.each(&:clear)
Sidekiq::RetrySet.new.clear
Sidekiq::ScheduledSet.new.clear
Sidekiq::DeadSet.new.clear

Josh Thompson about blog ☕️ with josh

Sidekiq in Practice

Article Table of Contents

Tips on Sidekiq Queues (Phil.tech) #