Shrinking

What is shrinking?

In this lesson, we’ll look at what happens when a property-based test fails. Property-based testing frameworks such as stream_data help you in those cases by shrinking the input that caused the test to fail and coming up with a simpler input that still causes the failure. Shrinking plays a fundamental role in the usefulness of property-based testing, and it’s important to understand how it works in order to take full advantage of the testing framework.

What a property failure looks like?

We’ve seen how property-based testing helps us write robust tests, in part by covering a larger space of inputs to feed to our code. However, we haven’t discussed what happens when the tests find an error in the code. To see what happens in that case, let’s come up with a sorting function that we know won’t work well.

The simplest one that comes to mind is the identity function, which is a function that takes one argument and returns that argument unchanged. This is a terrible sorting function because, well, it doesn’t do anything to the input list. However, this sort function still holds at least one of the two properties we’ve been testing: the length of the sorted list is the same as the length of the input list. That makes sense since it’s the same list. However, the identity function definitely doesn’t hold the property of the output list being sorted. Let’s write this up in a property and see what happens when it fails. Take a look at the bad sort test in the file property_based_testing/sorting/test/bad_sort_test.exs below: