Configuration

Operator error

Every piece of production-class software has scads of configurable properties containing:

• Hostnames
• Port numbers
• Filesystem locations
• ID numbers
• Magic keys
• Usernames
• Passwords
• Lottery numbers

Get any of these properties wrong and the system is broken. Even if the system seems to work most of the time, it could break at 1 a.m. when Daylight Saving Time kicks in. “Configuration” suffers from hidden linkages and high complexity, two of the biggest factors leading to operator error. This puts the system at risk because configuration is part of the system’s user interface. It’s the interface used by one of its most overlooked constituencies: the developers and operators who support it. Let’s look at some design guidelines for handling instance-level configuration.

Configuration files

The configuration is essentially a file or set of files the instance reads at startup. Configuration files may be buried deep in the directory structure of the codebase, possibly in multiple directories. Some of them represent basic application plumbing like API routes. Others need to change per environment.

Because the same software runs on several instances, some configuration properties should probably vary per machine. Keep these properties in separate places so nobody ever has to ask, “Are those supposed to be different?”

We don’t want our instance binaries to change per environment, but we do want their properties to change. That means the code should look outside the deployment directory to find per-environment configurations. These files contain the most sensitive information in the entire enterprise: production database passwords. They need to be protected from tampering and prying eyes. That leads us to another great reason to keep per-environment configuration out of the source tree: version control. Sooner or later, we’ll accidentally commit a production password to version control. GitHub currently shows 288,093 commits with the title “Removed password.” Tomorrow that number will be higher.

That’s not to say you should keep configurations out of version control altogether. Just keep them in a different repository than the source code. Lock it down to only the people who should have access, and make sure to have controls ( processes, procedures, and people following up on them) to grant and revoke access to those configurations.