Generalizing Test Examples

Generalizing test examples

Modeling tends to work well, as long as it is possible to write the same program multiple times, and as long as one of the implementations is so simple, it is obviously correct. This is not always practical, and sometimes not possible, so we need to find better properties. That’s significantly harder than finding any one property, which can already prove difficult and requires a solid understanding of the problem space.

Note: A good trick for finding a property is to start by writing a regular unit test and then abstract it away. We can take the common steps to come up with all individual examples and replace them with generators.

We will stick to our example of the biggest/1 function of the previous lesson. We already know that the biggest function is as reliable as the list:sorts/1 and list:last/1 functions. So, let’s write some test cases to demonstrate that they work as expected. Let’s see how we can write a property for lists:last/1. This function is so simple that we can consider it to be unquestionable. For this kind of function, traditional unit tests are usually a good fit since it’s easy to come up with examples that should be significant. We can also transform the example into a property. After all, if we can get a property to do the work for us, we’ll have thousands of examples instead of the few we’d come up with, and that’s objectively better coverage.

Let’s look at a few examples for the lists:last/1 function, and then generalize them into a property:

last_test() -> 
    ?assert(-23 =:= lists:last([-23])),
    ?assert(5 =:= lists:last([1,2,3,4,5])), 
    ?assert(3 =:= lists:last([5,4,3])).

The steps to write this test are:

1. Construct a list by picking a bunch of random numbers one by one.
   • Pick the first number.
   • Pick the second number.
   • …
   • Pick the last number.
2. Take note of the last number in the list as the expected result.
3. Check that the value returned is the same as the expected result.

Because the last substep of 1. (“Pick the last number.”) is what we want to focus on, we can separate it from the other substeps with some clever generator usage. If we group all of the initial substeps into a list and isolate the last one, we get something like {list(number()), number()}. Let’s see how that would look in a property.