Overview

We saw how we could run chaos experiments limited to specific applications or specific conditions. We were, for example, removing instances of an application, and that was partly randomized. We were terminating random Pods, but they were still tied to a specific application. Similarly, we were messing with networking but, again, limited and filtered by a particular app. We were destroying nodes as well, but they were those where specific applications were running.

Now we’re ready to spice it up a bit. We’re going to make our experiments even more random. Instead of being limited to destructions and interruptions of specific components, we’re going to start affecting completely random things. And that will bring us an additional level of insight that we might not get when we are limited to specific applications. We’ll explore what happens when we exercise, more or less, random interruptions. For that, we will need to make some changes to our experiments if we are not going to limit them to a single application. We could start affecting the system on the cluster level, but that might be too big of a jump.

We’ll start by trying to figure out how to affect random resource on the Namespace level. After that, we might move onto the cluster level. Through such goals, we might discover that the probes and steady-state hypotheses defined in Chaos Toolkit might not be sufficient. How do we know what the state that we want to observe is if that state is much bigger than a single application is? To be successful at a larger scale, we might need to bring additional tools into the mix. We might want to re-evaluate how we observe the state and how we’ll get notified if something is wrong inside our cluster after doing completely random destruction.

All in all, we’ll explore how to make chaos experiments more destructive, more random, and more unpredictable.

---

In the next lesson, we will be provided with a gist that contains all the commands for this chapter.