Black Box model in Testing Periodic Actions

The GenServer and its test in the previous lesson both work. However, there’s a problem with the test:

• It’ll happily pass if we decide to refactor our GenServer to send two or more messages when it ticks.

This happens because we use Mox.stub/3, which allows us to call the stub function as many times as possible and doesn’t assert how many times it’s called. Let’s try to fix this.

Compromising the black-box model

Mox provides the perfect function for what we want: Mox.expect/4. We can use expect/4 to ensure that the send_sms/2 function is called exactly once. However, our problem persists because of the nature of our test subject.

Since the GenServer performs its action periodically, we don’t know how many times it’ll tick in our tests. For example, if our machine is particularly slow, then the GenServer might tick a few times during the course of a single test if we use a fast interval as we did in the previous lesson. So the problem is in the foundations of our test.

As it turns out, it’s pretty hard to escape the hole we find ourselves in.

Note: In this course, we always try to strictly follow testing principles such as treating application code as a black box, and avoiding changes to application code just for the sake of making testing easier. However, we have already bent the rules slightly before. For example, reading the @twilio_module attribute at compile time to swap the Twilio interface double during testing is surely a change to apply code to favor testing. In this case, we’ll take a similar but slightly more invasive approach and make a logic change to SoggyWaffle.WeatherChecker to make testing easier. We’re walking the line between clean tests that are practical, while being effective at the same time.

The fundamental problem we have is that our GenServer performs a periodic action in the background that, once set off, keeps repeating until we stop the GenServer itself. What if we didn’t act periodically but on-demand?

Our current SoggyWaffle.WeatherChecker doesn’t support that, but it’s something we can change. We’ll add support for a :mode option when starting the GenServer. This option controls whether the GenServer will behave as it does now and perform the tick periodically (:periodic mode) or whether we’ll trigger each tick manually (:manual mode).