Creating AWS Volumes

Exploring the Options to Persist State

If we want to persist a state that will survive even server failures, we have two options to choose.

Local storage

We could, for example, store data locally and replicate it to multiple servers. This way, a container could use local storage, knowing that the files are available on all the servers. Such a setup would be too complicated if we’d like to implement the process ourselves. We could use one of the volume drivers for that. However, we’ll opt for a more commonly used method to persist the state across failures. We’ll use external storage.

External storage

Since we’re running our cluster in AWS, we can choose between S3, Elastic File System (EFS), and Elastic Block Store.

S3 is meant to be accessed through its API and is not suitable as a local disk replacement. This leaves us with EFS and EBS.

Elastic File System (EFS)

Elastic File System (EFS) has a distinct advantage: it can be mounted to multiple EC2 instances spread across multiple availability zones. It is the closest we can get to fault-tolerant storage. Even if a whole zone (data center) fails, we’ll still be able to use EFS in the rest of the zones used by our cluster. However, this comes at a cost. EFS introduces a performance penalty. It is, after all, a network file system (NFS), which entails higher latency.

Elastic Block Store (EBS)

Elastic Block Store (EBS) is the fastest storage we can use in AWS. Its data access latency is very low, therefore making it the best choice when performance is the primary concern. The downside is availability. It doesn’t work in multiple availability zones. Failure of one will mean downtime, at least until the zone is restored to its operational state.

Our Choice

We’ll choose EBS for our storage needs. Jenkins depends heavily on IO, and we need data access to be as fast as possible. However, there is another reason for such a choice. EBS is fully supported by Kubernetes. EFS will come but it is still in the experimental stage. As a bonus advantage, EBS is much cheaper than EFS.

Given the requirements and what Kubernetes offers, the choice is obvious. We’ll use EBS, even though we might run into trouble if the availability zone where our Jenkins will run goes down. In such a case, we’d need to migrate EBS volume to a healthy zone. There’s no such thing as a perfect solution.

We are jumping ahead of ourselves. We’ll keep Kubernetes aside for a while and concentrate on creating an EBS volume.

Creating an EBS volume

Each EBS volume is tied to an availability zone. Unlike EFS, the Elastic Block Store cannot span multiple zones. So, the first thing we need to do is to find out which zones worker nodes are running in.

Looking into the description

We can get the above information by describing the EC2 instances belonging to the security group nodes.devops23.k8s.local.