Defining Resource Quotas for a Namespace

The problem

Resource defaults and limitations are a good first step towards preventing malicious or accidental deployment of Pods that can potentially produce adverse effects on the cluster. Still, any user with the permissions to create Pods in a namespace can overload the system. Even if max values are set to some reasonably small amount of memory and CPU, a user could deploy thousands or even millions of Pods and consume all the available cluster resources. Such an effect might not be even produced out of malice but accidentally.

A Pod might be attached to a system that scales it automatically without defining upper bounds, and before we know it, it might scale to too many replicas. There are also many other ways things might get out of control.

The solution

We need to define namespace boundaries through quotas.

With quotas, we can guarantee that each namespace gets its fair share of resources. Unlike LimitRange rules that are applied to each container, ResourceQuota defines namespace limits based on aggregate resource consumption.

We can use ResourceQuota objects to define the total amount of compute resources (memory and CPU) that can be spent in a namespace. We can also use it to limit storage utilization or the number of objects of a certain type that can be created in a namespace.

Devising a plan

Let’s look at the cluster resources we have in our cluster. It is small, and it’s not even a real cluster. However, it’s the only one we have (for now). For now, we’ll consider it as a real cluster…

Our cluster has 2 CPUs and 2 GB of memory. Now, let’s say that this cluster serves only development and production purposes. We can use the default namespace for production and create a dev namespace for development. We can assume that the production should consume all the resources of the cluster except those given to the dev namespace, which, on the other hand, should not exceed a specific limit.

The truth is that with 2 CPUs and 2 GB of memory, there isn’t much we can give to developers. Still, we’ll try to be generous. We’ll give them 500 MB and 0.8 CPUs for requests. We’ll allow occasional bursts in resource usage by defining limits of 1 CPU and 1 GB of memory. Furthermore, we might want to limit the number of Pods to ten. Finally, as a way to reduce risks, we will deny developers the right to expose node ports.

This seems like a decent plan, no? Let’s move on and create the quotas we discussed.

Defining the quotas

Let’s look at the dev.yaml definition: