A Different Way to Unit Test a Sidekiq Job

03 Aug 2020 - Mangesh Tamhankar

I’ve always unit tested Sidekiq jobs in more or less the same way. Let’s take a look at a simple example:

class HardWorker
  include Sidekiq::Worker

  def perform(person_id)
    person = Person.find(person_id)
    person.update(processed: true)
  end
end

Given a class like this, I might have written a test like so:

RSpec.describe HardWorker do
  subject(:worker) { described_class.new }
  let(:person) { create(:person) }

  it 'processes a person' do
    worker.perform(person.id)

    expect(person.reload.processed).to eq true
  end
end

Cool. Now, wherever I call this worker, I might have written a test that made an assertion like:

expect(HardWorker).to receive(:perform_async).with(person_1.id)

or if I wanted to be sure that the job was queued, or take into account other methods of enqueueing the job (e.g. perform_in or a Sidekiq::Client.push_bulk), I might write:

expect{
  task
}.to change(HardWorker.jobs.size).by(1)

This all works. Everyone is happy. Let’s go home.

But wait. What if I had written my HardWorker perform method with a keyword argument:

def perform(person_id:)
  person = Person.find(person_id)
  person.update(processed: true)
end

The corresponding unit test can be updated to call worker.perform(person_id: person.id). Okay, this seems fine. I can update callers to include the keyword argument. All of my unit tests still pass… but there’s a problem. Sidekiq does not support keyword arguments because keyword arguments do not survive the serialization/deserialization round trip.

I have written and deployed code like this exactly twice! Luckily, neither had huge consequences. After the second time, I added a Rubocop linter to prevent someone from repeating the mistake in our project. But what if there were also a different way to write my unit test that would have caught this? Let’s take a look:

RSpec.describe HardWorker do
  let(:person) { create(:person) }

  it 'processes a person' do
    expect {
      described_class.perform_async(person_id: person.id)
      described_class.drain
    }.not_to raise_error
  end
end

Instead of directly calling the #perform method on an instance, we can call .perform_async on the class to mimic how we actually expect our class to be used in the real world. We can then call .drain to run the jobs.

Now, our test is sending the arguments through the serialization roundtrip.

In this contrived example, I’ve simply asserted that this process should not raise an error, but if I run it, I get the following:

  expected no Exception, got #<ArgumentError: wrong number of arguments (given 1, expected 0; required keyword: person_id)> with backtrace:

Awesome! Had I written my tests this way, I would have caught my problem before pushing this faulty code to production. Are we breaking some of the principles of unit testing by involving the serialization process? Maybe a little bit, but the tradeoff certainly has its benefits. Linters are not perfect, for one, and keyword arguments are not the only things that fail to survive Sidekiq’s serialization/deserialization process (e.g. symbols, Date/Time, or other complex Ruby objects). Sidekiq Best Practices make note of this, but sometimes you just need a little nudge or reminder of it 😉

A quick shoutout to Lucy Lufei, who originally introduced me to this paradigm.