Coding With Tests

Learn about the fundamentals of test-driven development, the primary testing objectives, and their patterns.

We'll cover the following

Test-driven development

Write tests first is the mantra of test-driven development. Test-driven development takes the untested code is broken code concept one step further and suggests that only unwritten code should be untested. We don’t write any code until after we have written the tests that will prove it works. The first time we run a test, it should fail, since the code hasn’t been written. Then, we write the code that ensures the test passes, and then write another test for the next segment of code.

Test-driven development can be fun; it allows us to build little puzzles to solve. Then, we implement the code to solve those puzzles. After that, we make a more complicated puzzle, and we write code that solves the new puzzle without unsolving the previous one.

There are two goals of the test-driven methodology. The first is to ensure that tests really get written. Secondly, writing tests first forces us to consider exactly how the code will be used. It tells us what methods objects need to have and how attributes will be accessed. It helps us break up the initial problem into smaller, testable problems, and then recombine the tested solutions into larger, also tested, solutions. Writing tests can thus become a part of the design process. Often, when we’re writing a test for a new object, we discover anomalies in the design that force us to consider new aspects of the software.

Testing makes software better. Writing tests before we release the software makes it better before the final code is written. All of the code examined in the book has been run through an automated test suite. It’s the only way to be absolutely sure the examples are rock-solid, working code.

Testing objectives

We have a number of distinct objectives for running tests. These are often called types of testing, but the word type is heavily overused in the software industry. We’ll look at only two of these testing goals:

  • Unit tests confirm that software components work in isolation. We’ll focus on this first, since Fowler’s Test Pyramid seems to suggest unit testing creates the most value. If the various classes and functions each adhere to their interfaces and produce the expected results, then integrating them is also going to work nicely and have relatively few surprises. It’s common to use the coverage tool to be sure all the lines of code are exercised as part of the unit test suite.

  • Integration tests confirm software components work when integrated. Integration tests are sometimes called system tests, functional tests, and acceptance tests, among others. When an integration test fails, it often means an interface wasn’t defined properly, or a unit test didn’t include some edge case that’s exposed through the integration with other components. Integration testing seems to depend on having good unit testing, making it secondary in importance.

The unit isn’t formally defined by the Python language. This is an intentional choice. A unit of code is often a single function or a single class. It can be a single module, also. The definition gives us a little flexibility to identify isolated, individual units of code.

While there are many distinct objectives for tests, the techniques used tend to be similar. All tests have a common pattern to them, and we’ll look at a general pattern of testing next.

Testing patterns

Writing code is often challenging. We need to figure out what the internal state of the object is, what state changes it undergoes, and determine the other objects it collaborates with. Throughout the course, we’ve provided a number of common patterns for designing classes.

Tests, in a way, are simpler than class definitions, and all have essentially the same pattern:

Press + to interact
GIVEN some precondition(s) for a scenario
WHEN we exercise some method of a class
THEN some state change(s) or side effect(s) will occur that we can
confirm

In some cases, the preconditions can be complex or perhaps the state changes or side effects are complex. They might be so complex that we have to break them into multiple steps. What’s important about this three-part pattern is how it disentangles the setup, execution, and expected results from each other. This model applies to a wide variety of tests. If we want to make sure the water’s hot enough to make another cup of tea, we’ll follow a similar set of steps:

  • GIVEN a kettle of water on the stove
  • AND the burner is off
  • WHEN we flip open the lid on the kettle
  • THEN we see steam escaping

This pattern is quite handy for making sure we have a clear setup and an observable result.

Let’s say we need to write a function to compute an average of a list of numbers, excluding None values that might be in the sequence. We might start out like this:

Press + to interact
def average(data: list[Optional[int]]) -> float:
  """
  GIVEN a list, data = [1, 2, None, 3, 4]
  WHEN we compute m = average(data)
  THEN the result, m, is 2.5
  """
  pass

We’ve roughed out a definition of the function, with a summary of how we think it should behave. The GIVEN step defines some data for our test case. The WHEN step defines precisely what we’re going to be doing. Finally, the THEN step describes the expected results. The automated test tool can compare actual results against the stated expectation and report back if the test fails. We can then refine this into a separate test class or function using our preferred test framework. The ways unittest and pytest implement the concept differ slightly, but the core concept remains in both frameworks. Once that’s done, the test should fail and we can start implementing the real code, given this test as a clear goal line we want to cross.

Some techniques that can help design test cases are equivalence partitioning and boundary value analysis. These help us decompose the domain of all possible inputs to a method or function into partitions. A common example is locating two partitions, valid data and invalid data. Given the partitions, the values at the boundaries of the partitions become interesting values to use in test cases.

We’ll start by looking at the built-in testing framework, unittest. It has a disadvantage of being a bit wordy and complicated looking. It has the advantage of being built-in and usable immediately; no further installs are required.