Using os/exec to Automate Local Changes

Automating the execution of tools that are local to the machine can provide a series of benefits to end users. The first of these is that it can reduce the toil that our team experiences. One of the primary goals for DevOps and Site Reliability Engineers (SRE) is to remove repetitive, manual processes. That time can be put to better use by reading a good book, organizing a sock drawer, or working on the next problem. The second benefit is to remove manual mistakes from a process.

It is easy to type the wrong thing or copy and paste something incorrectly. And finally, it is the core underpinning of operating at scale. Automating locally can be combined with other techniques detailed in the book to make changes at a large scale.

The automation life cycle generally comes in three stages, moving from manually doing work to automation, as follows:

1. The first stage revolves around the manual execution of commands by an experienced engineer. While this is not automation itself, this starts a cycle that ends with some type of automation.

2. The second stage usually revolves around writing those stages down in order to document the procedure, to allow more than one person to share the workload. This might be a method of procedure (MOP) document, though more commonly, it is a bunch of notes that we spend an hour looking for. We highly recommend a central place to store these such as a wiki or markdown in a git repository.

3. The third stage is usually a script to make the task repeatable.

Once a company gets larger, these stages are usually condensed into developing a service to handle the task in a fully automated way when a need for it is identified. A good example of this might be deploying pods on a Kubernetes cluster or adding a new pod configuration to our Kubernetes config. These are driven by calling command-line applications such as kubectl and git.

These types of jobs start manually; eventually, they are documented and finally automated in some way. At some point, this might move into a continuous integration/continuous deployment (CI/CD) system that handles this for us.

The key to automating tooling locally is the os/exec package. This package allows for the execution of other tools and control of their STDIN/STDOUT/STDERR streams.

Let's take a closer look.

Determining the availability of essential tools

When writing an application that calls other applications on a system, it is critical to determine if the tools needed are available on the system before we start executing commands. Nothing is worse than being partway through a procedure to find that a critical tool is missing.

The exec package provides the LookPath() function to help determine if a binary exists. If only the name of the binary is provided, the PATH environmental variable is consulted, and those paths will be searched for the binary. If a / is in the name, only that path will be consulted.

Let's ...