Gap Between Documentation and Actual Implementation

Robustness principle

If you have machine-readable specifications for our message formats, we should be able to verify these properties by analyzing the new specification relative to the old spec. A tough problem arises that we need to address when applying the Robustness Principle, though. There may be a gap between what we say our service accepts and what it really accepts.

JSON payload example

For instance, suppose a service takes JSON payloads with a url field. We discover that the input is not validated as a URL, but just received as a string and stored in the database as a string. We want to add some validation to check that the value is a legitimate URL, maybe with a regular expression. Bad news: the service now rejects requests that it previously accepted. That is a breaking change.

But wait a minute! The documentation said to pass in a URL. Anything else is bad input and the behavior is undefined, meaning it could do absolutely anything. As soon as the service went live, its implementation became the de facto specification.

It’s common ...