Property-Based Testing with PropEr in Elixir/

...

Refining the Tests

Understand how refining the tests we have written for the application will yield better results.

We'll cover the following...

Code
Understanding the bug
Refining the tests

Order of failures:
Property tests are probabilistic. We may find different issues, or the same issues in a different order, than those in this text. This is normal, and rerunning the property a few times may yield different errors each time.
The first error in this section can’t be reproduced easily in Elixir, where the string() generator is not exported by PropCheck. Instead, utf8() is used. However, utf8() more frequently returns well-formed Unicode data that doesn’t cause the failure.

Code

Let’s start off with the code and make fixes to it as we go through the lesson.

Note: To run an updated version of the code, press run, and then call mix test in the terminal that opens.

defmodule BookShim do
  def add_book_existing(isbn, title, author, owned, avail) do
    Bookstore.DB.add_book(isbn, title, author, owned, avail)
  end
  def add_book_new(isbn, title, author, owned, avail) do
    Bookstore.DB.add_book(isbn, title, author, owned, avail)
  end

  def add_copy_existing(isbn), do: Bookstore.DB.add_copy(isbn)
  def add_copy_new(isbn), do: Bookstore.DB.add_copy(isbn)

  def borrow_copy_avail(isbn), do: Bookstore.DB.borrow_copy(isbn)
  def borrow_copy_unavail(isbn), do: Bookstore.DB.borrow_copy(isbn)
  def borrow_copy_unknown(isbn), do: Bookstore.DB.borrow_copy(isbn)

  def return_copy_full(isbn), do: Bookstore.DB.return_copy(isbn)
  def return_copy_existing(isbn), do: Bookstore.DB.return_copy(isbn)
  def return_copy_unknown(isbn), do: Bookstore.DB.return_copy(isbn)

  def find_book_by_isbn_exists(isbn) do
    Bookstore.DB.find_book_by_isbn(isbn)
  end
  def find_book_by_isbn_unknown(isbn) do
    Bookstore.DB.find_book_by_isbn(isbn)
  end

  def find_book_by_author_matching(author) do
    Bookstore.DB.find_book_by_author(author)
  end
  def find_book_by_author_unknown(author) do
    Bookstore.DB.find_book_by_author(author)
  end

  def find_book_by_title_matching(title) do
    Bookstore.DB.find_book_by_title(title)
  end
  def find_book_by_title_unknown(title) do
    Bookstore.DB.find_book_by_title(title)
  end
end

Refine the tests

Understanding the bug

Running the tests now fails on Unicode values of 0x00, meaning that the protocol likely uses null-terminated strings, which causes failures. Anyone using this PostgreSQL driver needs to protect themselves against such strings.

For now, we’ll assume that those invalid Unicode values should either be filtered somewhere else or that a bunch of other properties or unit tests will handle these, and we’ll instead work around it by making sure our generators don’t generate that data anymore. In fact, we’ll find a lot of other troublesome characters: % and _ will influence PostgreSQL search in ways that string:find/2 wouldn’t, and \ will mess with escaping in SQL whereas our model won’t care.