The Relation Between Application and Environment Pipelines

Keep in mind that I am aware we haven’t gone into detail on the application pipeline, that’s coming soon. Right now, we’ll focus on the overall flow between the two.

The flow from a commit to the master branch to deployment to the staging environment

Everything starts with a push into the master branch of an application repository (e.g., go-demo-6). That push might be direct or through a pull request. For now, what matters is that something is pushed to the master branch.

A push to any branch initiates a webhook request to Jenkins X. It does not matter much whether the destination is Jenkins itself, Prow, or something else. We haven’t gone through different ways we can define webhook endpoints. What matters is that the webhook might initiate activity that performs a set of steps defined in jenkins-x.yml residing in the repository that launched the process. Such an activity might do nothing if jenkins-x.yml ignores that branch, it might execute only a fraction of the steps, or it might run all of them. It all depends on the branch filters. In this case, we’re concerned with the steps defined to run when a push is done on the master branch of an application.

From a very high level, a push from the master branch of an application initiates a build that checks out the code, builds binaries (e.g., container image, Helm chart), makes a release and pushes it to registries (e.g., container registry, Helm charts registry, etc.), and promotes the release. This last step is where GitOps principles are more likely to clash with what you’re used to doing. More often than not, a promotion would merely deploy a new release to an environment. We’re not doing that because we’d break at least four of the rules we defined.

If the build that was initiated through a Webhook of an application repository results in deployment, that change would not be stored in Git.

• We could not say that Git is the only source of truth.
• That deployment would not be tracked.
• The operation would not be reproducible.
• Not everything would be idempotent.
• Finally, we would also break the rule that information about all the releases must be stored in environment-specific repositories or branches.

Truth be told, we could fix these issues by pushing a change to the environment-specific repository after the deployment, but that would break the rule that everything must follow the same coding practices. Such a course of action would result in an activity that was not initiated by a push of a change to Git, and would not follow whichever coding practices we decided to follow. We have to follow all the rules, but keep in mind that the order matters as well. Simply put, we push a change to Git, and that ends with a change of the system, not the other way around.

Taking all that into account, the only logical course of action is for the promotion steps in the application pipeline to make a push to a branch of the environment-specific repository. Given that we choose to promote to the staging environment automatically, it should also create a pull request, it should approve it, and it should merge it to the master branch automatically. That is an excellent example of following a process, even when humans are not involved.

At this point, you might be asking yourself:

"What is the change to the environment-specific repository pushed by the application-specific build?"

If you paid attention to the contents of the requirements.yaml file, you should already know the answer. Let’s output it one more time as a refresher.