Testing rules

Below are the general rules that we should follow during testing:

Rules

• You must write the test as soon as you write the code. Writing the test some days after writing the code is useless because it’s easy to forget the choices you made (choices that you need to test).

• The test must fail before you write the code and succeed when you have written the code. Otherwise, you can not be sure whether the test is correctly testing your code and that it is your code that enables the test to succeed. If you are fixing a bug, you want to be sure that you did not accidentally, systematically reproduce it.

• A failing test is a good thing! A failing test protects us from a regression! The important thing to remember is that the tests should not fail randomly. False-negative tests are the worst ones, you do not know why they fail and you stop trusting them. Obviously, the opposite is important too: tests must fail if the app does not work, otherwise, what are they testing?

• Tests must be simple. Simple to read, understand, and change. Tests should be simple scripts that perform some checks for you. While the point of writing tests is refactoring code, you don’t want to refactor complex or unclear tests. Remember: debugging a failing test is much more complicated than debugging an application.

• Do not DRY the test code too early. While testing, you need to change more or less every line of the test code to simulate the different cases. Applying DRY when not necessary increases the test complexity and forces you to add many conditions to the DRYed code.

• Avoid conditional tests. The test code must be straightforward.

• A failing test must give useful feedback. It must tell you what failed and why, without re-launching or debugging it.

• Tests must be deterministic. If you run them a hundred times, the result must be the same. If they are not deterministic, you must think about what influences them and write better tests. Deterministic tests mean that you should not rely on their execution order. Otherwise, you end up with brittle tests that need to be run in a specific order.

• Tests must be fast. While you should never sacrifice speed for reliability, the faster the tests are, the faster you have feedback. Therefore, the faster the test, the sooner you discover regressions.

• Never do white-box testing. You have to test APIs, contracts, and behaviors, not internal details. You must test as the consumer, not as the author.

• Tests must be independent of each other. Tests must not share any state.

• Never manually interact with the tests. While manual interaction makes performing functional tests easier, tests must not depend on human changes.

• Following the black-box testing rule, an E2E test can not provide useful feedback for a single function, and a unit test can not tell you if an app works. Different test types provide different feedback and have different costs, get used to that.

• While code coverage enables us to find what has not been tested yet, it is not an end in itself.

• If you don’t feel confident about your application even if it passes your tests, think twice about your tests. Why don’t they give you confidence? How could you improve them?

• Don’t underestimate “skip” and “only test” utilities. These utilities are super useful to run a portion of the tests and remove unnecessary noise.

• Do not test how external services work, but how they are consumed. Do not test third-party scripts and libraries.

• Do not test native APIs.

• Remember that testing and Test-Driven Development (TDD) are two different things. TDD goes beyond testing and requires a particular context to be practiced. You can test everything without ever applying TDD.

Illustration