Negative Testing: Broad Properties

Negative tests

An interesting aspect of the code and tests we have written so far is that the list of items being bought is generated through the list of supported items. Designing the test case from the price list shows a lack of tests looking for unexpected use cases. Our tests are positive, happy-path tests, validating that everything is operating as it should. Negative tests, by comparison, try to specifically exercise underspecified scenarios to find what happens in less happy paths.

Our current properties are good. We don’t want to modify them or make them more complex. Instead, we’ll write a few broad properties that test more general properties of the code and see if the requirements we had in place are actually consistent. On their own, broad (and often vague) properties are not too useful, but they serve as great anchors to specific properties. The broader the properties, the lower the chances are that we’ll be searching only for expected problems, as is usually done with traditional tests.

Broad properties

Let’s get started with broad properties and use them to test our application.

Starting off

Let’s start with an extremely broad property:

“The checkout:total/3 function should return an integer and not crash.”

Let’s take a look at what this would look like.

We have the prop_expected_results property added to the code widget below in the test/prop_checkout.erl file at line 25.

This also introduces a new generator, lax_lists(), defined as broadly as possible at first.

Let’s take a look at the property and the generator to see it working.