Testing a Circuit Breaker

Understanding the system

Circuit breakers are one of the most interesting concepts in fault isolation for reliable systems. They are used to account for errors that happen over time in a part of the system. If the frequency at which the errors happen is considered too high compared to the successful cases, the breaker is tripped.

Once tripped, further calls automatically fail before getting a chance to reach the subsystem gated by the breaker. The idea is that failures tend to be costly and take a lot of time, and a failing system under heavy stress is even harder to get back into a usable state. The circuit breaker helps us prevent the client from doing work that would result in expensive failures. Instead, we get cheap failures that allow the subsystem to recuperate under less stress.

For this example, we’ll use the circuit_breaker library from Klarna and give it tests. But first, let’s see how it works.

Understanding circuit_breaker

Let’s start by looking at how the library works. After that, we can move on to writing tests for it.

The library can be used as a wrapper over a given function call. This wrapper looks at the return values ...