Property-Based Testing with PropEr in Erlang/

...

/

Refining the Tests

Refining the Tests

Understand how refining the tests we have written for the application will yield better results.

We'll cover the following...

Order of failures:
Property tests are probabilistic. If you are following along, you may find different issues, or the same issues in a different order, than those in this text. This is normal, and rerunning the property a few times may yield different errors each time.

Code

Let’s start off with the code and make fixes to it as we go through the lesson.

Note: To run an updated version of the code, press “Run”, and then call rebar3 proper in the terminal that opens.

-module(book_shim).
-compile(export_all).

add_book_existing(ISBN, Title, Author, Owned, Avail) -> 
    bookstore_db:add_book(ISBN, Title, Author, Owned, Avail).
add_book_new(ISBN, Title, Author, Owned, Avail) -> 
    bookstore_db:add_book(ISBN, Title, Author, Owned, Avail).

add_copy_existing(ISBN) -> bookstore_db:add_copy(ISBN). 
add_copy_new(ISBN) -> bookstore_db:add_copy(ISBN).

borrow_copy_avail(ISBN) -> bookstore_db:borrow_copy(ISBN). 
borrow_copy_unavail(ISBN) -> bookstore_db:borrow_copy(ISBN). borrow_copy_unknown(ISBN) -> bookstore_db:borrow_copy(ISBN).

return_copy_full(ISBN) -> bookstore_db:return_copy(ISBN). 
return_copy_existing(ISBN) -> bookstore_db:return_copy(ISBN). 
return_copy_unknown(ISBN) -> bookstore_db:return_copy(ISBN).

find_book_by_isbn_exists(ISBN) -> bookstore_db:find_book_by_isbn(ISBN). 
find_book_by_isbn_unknown(ISBN) -> bookstore_db:find_book_by_isbn(ISBN).

find_book_by_author_matching(Author) -> 
    bookstore_db:find_book_by_author(Author).
find_book_by_author_unknown(Author) -> 
    bookstore_db:find_book_by_author(Author).

find_book_by_title_matching(Title) -> 
    bookstore_db:find_book_by_title(Title).
find_book_by_title_unknown(Title) -> 
    bookstore_db:find_book_by_title(Title).
Refine the tests

Understanding the bug

Running the tests then fails on Unicode values of 0x00, meaning that the protocol likely uses null-terminated strings, which causes failures. Anyone using this PostgreSQL driver needs to protect themselves against such strings.

For now, we’ll assume that those invalid Unicode values should either be filtered somewhere else or that a bunch of other properties or unit tests will handle these, and we’ll instead work around it by making sure our generators don’t generate that data anymore. In fact, we’ll find a lot of other troublesome characters: % and _ will influence PostgreSQL search in ways that string:find/2 wouldn’t, and \ will mess with escaping in SQL whereas our model won’t care.

Refining the tests

We have to ask ourselves whether what we want to test here is the minutiae of the SQL string handling, or whether these strings are a tool we use to validate the state transitions of our stateful model. This is the point where you can decide to split your property into two distinct properties:

  1. One property tests the search patterns explicitly to see if there are risky things happening with special
...