Negative Testing: Calibrating Negative Properties
Explore how to calibrate negative properties in property-based testing by gathering test statistics and adjusting generators. Understand techniques to improve test coverage between passing and failing cases. Learn to identify and fix bugs such as invalid input handling within Erlang code using PropEr.
We'll cover the following...
Getting started
Gathering statistics is the easiest and best tool we have to check whether a property is good. Let’s take a look again at prop_expected_result(). This time, we’ll look into the type of result we get. Right now, the property has two valid cases:
- The list of items are all valid and it will pass.
- At least some items are missing from the price list and it will fail.
If we use the collect/2 function to gather statistics, we get the following:
===> Testing prop_checkout:prop_expected_result()
....................................................................................................
OK: Passed 100 test(s).
91% prices_missing
9% valid
The vast majority of all failing test cases only exercise the one failing case we have identified, the one where an item isn’t on the price list. ...