Evolution of Jenkins Jobs and How We Got the YAML-Based Format

⚠️ The examples in this chapter work only with serverless Jenkins X.

The Jenkins X pipeline extension model is, in my opinion, one of the most exciting and innovative improvements we recieved with Jenkins X. It allows us to focus on what really matters in our projects and ignore the steps that are common to others. Understanding the evolution of Jenkins pipelines is vital if we are to adopt the extension model. Before we dive into extensions, we need to understand how pipelines evolved over time.

Freestyle jobs

When Jenkins appeared, its pipelines were called Freestyle jobs. There was no way to describe them in code, and they were not kept in version control. We were creating and maintaining those jobs through Jenkins UI by filling input fields, marking checkboxes, and selecting values from drop-down lists. The results were impossible-to-read XML files stored in the Jenkins home directory. Nevertheless, that approach was so great (compared to what existed at the time) that Jenkins become widely adopted overnight.

But, that was many years ago and what was great over a decade ago is not necessarily as good today. As a matter of fact, Freestyle jobs are the antithesis of the types of jobs we should be writing today. Tools that create code through drag-and-drop methods are extinct. Not having code in version control is a cardinal sin. Not being able to use our favorite IDE or code editor is unacceptable. Hence, the Jenkins community created Jenkins pipelines.

Jenkins pipelines

Jenkins pipelines are an attempt to rethink how we define Jenkins jobs. Instead of the click-type-select approach of Freestyle jobs, pipelines are defined in Jenkins-specific Groovy-based domain-specific language (DSL) written in Jenkinsfile and stored in code repositories together with the rest of the code of our applications. That approach managed to accomplish many improvements when compared to Freestyle jobs.

A single job could define the entire pipeline, the definition could be stored in the code repository, and we could avoid part of the repetition through the usage of Jenkins Shared Libraries. But, for some users, there was a big problem with Jenkins pipelines.

The problem with Jenkins pipelines

Using Groovy-based DSL was too big of a jump for some Jenkins users. Switching from click-type-select with Freestyle jobs into Groovy code was too much. We had a significant number of Jenkins users with years of experience in accomplishing results by clicking and selecting options without knowing how ...