Ten Commandments of GitOps Applied to Continuous Delivery

Instead of listing someone else’s rules, we’ll try to deduce them ourselves. So far, we have only one, and that is the most important rule that is likely going to define the rest of the brainstorming and discussion.

1. Git is the only source of truth

The one rule to rule them all is that Git is the only source of truth.

It is the first and most important commandment. All application-specific code in its raw format must be stored in Git. By code, I mean not only the code of your application, but also the tests, configuration, and everything else that is specific to that app or the system in general.

I intentionally said that it should be in raw format because there is no benefit of storing binaries in Git. That’s not what it’s designed for.

The real question is why we want those things? For one, good development practices should be followed. Even though we might disagree on which practices are good, and which aren’t, they are all levitating around Git.

• If you’re doing code reviews, you’re doing it through Git.
• If you need to see the change history of a file, you’ll see it through, Git.

If you find a developer doubting whether the code should be in Git (or some other code repository), please make sure that they’re isolated from the rest of the world because you just found a specimen of endangered species. There are only a few left, and they are bound to be extinct.

2. Everything must be tracked, actions should be reproducible, and idempotent

While there is no doubt amongst developers about where to store the files they create, that’s not necessarily true for other types of experts. I see testers, operators, and people in other roles that are still not convinced that’s the way to go and whether absolutely everything should be documented and stored in Git.

As an example, I still meet operators who run ad-hoc commands on their servers. As we all know, ad-hoc commands executed inside servers are not reliably reproducible, they are often not documented, and the result of their execution is often not idempotent.

So, let’s create a second rule: everything must be tracked, every action must be reproducible, and everything must be idempotent.

• If you only run a command instead of creating a script, your activities are not documented.
• If you did not store it in Git, others will not be able to reproduce your actions.

Finally, that script must be able to produce the same result no matter how many times we execute it. Today, the easiest way to accomplish that is through declarative syntax. More often than note, that would be YAML or JSON files that describe the desired outcome, instead of imperative scripts.

Let’s take installation as an example. If it’s imperative (install something), it will fail if that something is already installed; it won’t be idempotent.

Every change must be recorded (tracked). The most reliable and easiest way to accomplish that is by allowing people only to push changes to Git. That and only that is the acceptable human action!

What that means:

• If we want our application to have a new feature, we need to write code and push it to Git.
• If we want it to be tested, we write tests and push them to Git, preferably at the same time as the code of the application.
• If we need to change a configuration, we update a file and push it to Git.
• If we need to install or upgrade OS, we make changes to files of whichever tool we’re using to manage our infrastructure, and we push them to Git.

It all boils down to the fact that you should push it to Git. What is more interesting is what we should NOT do.

You are not allowed to add a feature of an application by changing the code directly inside the production servers.

It doesn’t matter how big or small the change is, it cannot be done by you, because you cannot provide a guarantee that the change will be documented, reproducible, and tracked. Machines are much more reliable when performing actions inside your production systems. You are their overlord, you’re not one of them. Your job is to express the desired state, not to change the system to comply with it.

3. Communication between processes must be asynchronous

The real challenge is to decide:

• How will that communication be performed?
• How do we express our desires in a way that machines can execute actions that will result in convergence of the actual state into the desired one?

An analogy

We can think of us as an aristocracy and the machines as servants. The good thing about an aristocracy is that there is no need to do much work. As a matter of fact, not doing any work is the main benefit of being a king, a queen, or an heir to the throne. Who would want to be a king if that means working as a car mechanic? No girl dreams of becoming a princess if that would mean working in a supermarket. Therefore, if being an aristocrat means not ...