System-Wide Transparency

System-wide transparency

Back in Transparency, we saw how individual instances can reveal their state. That’s the start of a total story about transparency. Now we’ll look at how to assemble a picture of system-wide health from the individual instances’ information.

The first place to start is by defining what we need from our efforts. When dealing with the system as a whole, two fundamental questions need to be answered:

1. Are users receiving a good experience?
2. Is the system creating the economic value we want?

Notice that the question, “Is everything running?” isn’t on that list. Even at a small scale, we should be able to survive periods where everything isn’t running. At scale, “partially broken” is the normal state of operation. It’s rare to find all instances running with no deployments or failures at any given moment.

Real-user monitoring

It is hard to deduce whether users are receiving a good experience from individual instance metrics.