Introduction

Is your software feature-complete?

You’ve worked hard on your project. It looks like all the features are actually complete, and most even have tests. You can breathe a sigh of relief. You’re done. Or are you? Does “feature complete” mean “production ready?” Is your system really ready to be deployed? Can it be run by operations and face the hordes of real world users without you? Do you worry about that sinking feeling that you’ll be faced with late-night emergency phone calls and alerts? It turns out there’s a lot more to development than just adding all the features. Software design as taught today is incomplete because it only talks about what systems should do. It doesn’t address what systems should not do.

Good systems

Good systems should avoid the following:

• Program Crash
• Hang
• Losing data
• Violating privacy
• Losing money

Too often, project teams aim to pass the quality assurance (QA) department’s tests instead of aiming for life in production.

That is, the bulk of our work probably focuses on passing tests. Even agile, pragmatic, automated testing is not enough to prove that software is ready for the real world. The stresses and strains of the real world like unpredictable real users, globe-spanning traffic, and virus-writing mobs from far-away countries go well beyond what we could ever hope to test for. But first, we’ll need to accept the fact that despite best laid plans, bad things will still happen. It’s always good to prevent them when possible, of course. But it can be downright fatal to assume that all possible bad events have been predicted and eliminated. Instead, we want to take action and prevent the ones we can, but make sure that the system as a whole can recover from whatever unanticipated, severe traumas might befall it.

Aiming for the right target

Most software is designed for the development lab or the testers in the QA department. It is designed and built to pass tests such as, “The customer’s first and last names are required, but the middle initial is optional.” It aims to survive the artificial realm of QA, not the real world of production. Software design today resembles automobile design in the early ’90s, disconnected from the real world. Cars designed solely in the cool comfort of the lab looked great in models and CAD systems. Perfectly curved cars gleamed in front of giant fans, purring in laminar flow. The designers inhabiting these serene spaces produced designs that were elegant, sophisticated, and clever, but were also fragile, unsatisfying, and ultimately short-lived. Most software architecture and design happens in equally clean, distant environments.

Real customers want more than just clever or good looking cars; they want to own a car designed for the real world. They want a car designed by somebody who knows that oil changes are always 3,000 miles late, that the tires must work just as well on the last sixteenth of an inch of tread as on the first, and that they will certainly, at some point, stomp on the brakes while holding an Egg McMuffin in one hand and a phone in the other.

When a system passes QA, can we say with confidence that it’s ready for production?

Simply passing QA tells us little about the system’s suitability for the next three to ten years of life. It could be the Toyota Camry of software, racking up thousands of hours of continuous uptime. Or it could be the Chevy Vega (a car whose front end broke off on the company’s test track) or the Ford Pinto (a car prone to blowing up when hit in just the right way). It’s impossible to tell from a few days or even a few weeks of testing what the next several years will bring.

Design for manufacturability

Product designers in manufacturing have long pursued “design for manufacturability,” the engineering approach of designing products such that they can be manufactured at low cost and high quality. Prior to this era, product designers and fabricators had different views than now. Designs thrown over the wall to production included screws that could not be reached, parts that were easily confused, and custom parts where off-the-shelf components would serve. Inevitably, low quality and high manufacturing cost followed.

We’re in a similar state today. We end up falling behind on the new system because we’re constantly taking support calls from the last half-baked project we shoved out the door. Our analog of “design for manufacturability” is “design for production.” We don’t hand designs to fabricators, but we do hand over finished software to IT operations. We need to design individual software systems, and the whole ecosystem of interdependent systems, to operate at low cost and high quality.