Negative Testing: Calibrating Negative Properties

Getting started

Gathering statistics is the easiest and best tool we have to check whether a property is good. Let’s take a look again at prop_expected_result(). This time, we’ll look into the type of result we get. Right now, the property has two valid cases:

1. The list of items are all valid and it will pass.
2. At least some items are missing from the price list and it will fail.

If we use the collect/2 function to gather statistics, we get the following:

===> Testing prop_checkout:prop_expected_result()
....................................................................................................
OK: Passed 100 test(s).
91% prices_missing
9% valid

The vast majority of all failing test cases only exercise the one failing case ...