Consequences of Avoiding Test Automation

This lesson discusses another obvious possibility regarding automated testing: simply not writing automated tests at all, perhaps not even testing at all. Is this viable?

The choice of not testing

Not testing at all is a choice we could make, and this might not be as silly as it sounds. If we define testing as verifying some outcome is achieved in its target environment, then things such as deep-space probes cannot truly be tested on Earth. At best, we’re simulating the target environment during our testing. Giant-scale web applications can rarely be tested with realistic load profiles. Take any large web application, launch a hundred million users at it—all doing invalid things—and see how most applications hold up. It’s probably not as well as developer testing suggested.

Development with reduced automation

There are areas of development where we might expect to see fewer automated tests:

• Extract, transform, and load (ETL) scripts for data migrations: ETL scripts are often one-off affairs, written to solve a specific migration problem with some data. It’s not always worth writing automated tests for these. We can perform manual verification on a similar set of source data instead.

• Front-end user interface work: Depending on the programming approach, it might be challenging to write unit tests for the front-end code. Whatever approach we take, assessing the visual look and feel cannot currently be automated. As a result, manual testing is often used against a candidate release of a user interface.

• Infrastructure as code (IaC) scripts: Our applications need to be deployed somewhere for them to run. A recent approach to deployment is to use languages such as Terraform to configure servers using code. This is an area that’s not yet simple to automate tests for.

So what actually happens when we abandon test automation, possibly not testing at all?

Impact of not testing

We might think that not testing at all is an option, but in reality, testing will always happen at some point. We can illustrate this with a timeline of the possible points at which testing can occur: