Fixtures and Mocks in Machine Learning Tests

Overview

Tests are considered reasonable if they frequently run, preferably each time we commit to our feature branch. Tests should not slow down the coding process of a programmer, so their design should be as minimal as possible: no complexity and computation, except minimal viable ones. Otherwise, the developer is tempted to disable their tests if they take more time than expected.

Here, lightweight fixtures come to the rescue. They can be patched instead of real models or data to speed up testing drastically. We refer to tests with these kinds of insertions as mock tests.

Shallow network as a fixture

Instead of using a large network, we can have a dummy replacement with similar properties (e.g., input and output dimensions), so we could use it in integration tests while keeping them fast enough.

Let’s look at the example below where a dummy Resnet50 backbone takes a 224x224x3 image as input and returns a 1x2048 feature vector as output.