Independent: The Third Quality of Valuable Tests

Independent

A test is independent if it doesn’t depend on any other tests or external data to run. An independent test suite gives the same results no matter how the tests are run and limits the scope of test failures to only tests that cover a buggy method.

Tests order

In contrast, a very dependent test suite could trigger failures throughout our tests from a single change in one part of an application. A clear sign that our tests are not independent is if we have test failures that happen only when the test suite is run in a particular order. In fully independent tests, the order in which they are run should not matter. Another sign is a single line of code-breaking multiple tests. We can also uncomment the generated line config.order = :random in the spec_helper.rb file to mandate random ordering of specs and to shake out potentially subtle test dependencies. Or we can use order --random on the RSpec command line to ensure random ordering.

Global data and test independence

The biggest impediment to independence in the test suite itself is the use of global data. Rails fixtures are not the only possible cause of global data in a Rails test suite, but they are a common cause. In a Rails context, it is somewhat less common to use a tool or third-party library in a setup and not tear it down.

RSpec bisecting

If we believe we have an intermittent test failure in RSpec triggered by order of testing, the rspec --bisect option will attempt to discover the minimal set of specs that trigger the failure. We’ll cover this in the chapter “Troubleshooting and Debugging”.

Test encapsulation

Outside the test suite, if the application code is not well-encapsulated, making the tests fully independent may be difficult or impossible.