Debugging Stateful Properties

Understanding the bug

The previously failing property is interesting. If we look at the counterexample it looks like a regular failure where our model or the system might have been wrong, but the shrinking imploded, and the source of failure isn’t obvious. In this lesson, we’ll revisit the shrinking mechanism of stateful properties to understand what goes on exactly and how to get PropEr to solve our problems for us.

Let’s first compare the initial failing command set and then compare it to the shrunken one. The initial failure was caused by the following sequence of events:

1. find_book_by_title_unknown(_) -> {ok, []}
2. borrow_copy_unknown(_) -> {error, not_found}
3. add_book_new(ISBNA, TitleA, AuthorA, 1, 1) -> ok
4. add_book_new(ISBNB, TitleB, AuthorB, 1, 1) -> ok
5. find_book_by_isbn_unknown(_) -> {ok, []}
6. add_copy_new(_) -> {error, not_found}
7. find_book_by_isbn_exists(ISBNA) -> {ok, [BookA]}
8. return_copy_full(ISBNA) -> ok

So we might have a legitimate bug since we successfully returned a copy of a book that was never borrowed, but the shrinking isn’t helping us since it appears it can’t prune the irrelevant operations out of the list without crashing. This isn’t great. How come shrinking failed us? The solution lies in the shrinking model.

In stateful properties, when we’re given a sequence of commands [A, B, C, D, E, F] that yields a failure, shrinking will be done by progressively removing commands from the sequence to see if a shorter one can cause the problem as shown in the following figure: