Tests Cannot Prevent Every Bug

A very old objection to testing of any kind is:

"We cannot catch every bug."

While this is certainly true, if anything, it means that we need more and better testing, not less. Let’s understand the motivations behind this one to prepare an appropriate response.

Understanding why people say tests cannot catch every bug

Straight away, we can agree with this statement. Tests cannot catch every bug. More precisely, it has been proven that testing in software systems can only reveal the presence of defects. It can never prove that no defects exist. We can have many passing tests, and defects can still hide in the places we haven’t tested.

This seems to apply in other fields as well. Medical scans will not always reveal problems that are too faint to notice. Wind tunnel tests for aircraft will not always reveal problems under specific flight conditions. Batch sampling in a chocolate factory will not catch every substandard sweet.