Terminating Random Application Instances
In this lesson, we will run an experiment using a CronJob to terminate random application instances, and then we will use the dashboards to figure out if it was successful or not.
Applying the CronJob
Let’s apply the CronJob that we just explored and see what’ll happen.
kubectl --namespace chaos apply --filename k8s/chaos/periodic-fast.yaml
Retrieving all CronJobs and Pods from the chaos
Namespace
Next, we’ll output all the CronJobs from the Namespace chaos
.
kubectl --namespace chaos get cronjobs
If the LAST SCHEDULE
is set to <none>
, you might need to wait for a while longer (up to two minutes), and re-run the previous command. Once the first Job is created, the output should be similar to the one that follows.
NAME SCHEDULE SUSPEND ACTIVE LAST SCHEDULE AGE
health-instances-chaos */2 * * * * False 1 8s 107s
Next, we’ll take a look at the Jobs created by that CronJob.
kubectl --namespace chaos get jobs
Just as we had to wait until the CronJob creates the Job, not we need to wait until the Job creates the Pod and the experiment inside it finishes executing. Keep re-running the previous command until the COMPLETIONS
column is set to 1/1
. The output should be similar to the one that follows.
NAME COMPLETIONS DURATION AGE
health-instances-chaos-... 1/1 93s 98s
From this moment on, the results I will present might differ from what you’ll observe on your screen. Ultimately, we’ll end up with the same result, even though the time you might need to wait for that might differ.
Finally, we’ll retrieve the Pods in the chaos
Namespace and check whether there was a failure.
kubectl --namespace chaos get pods
The output is as follows.
NAME READY STATUS RESTARTS AGE
health-instances-chaos-... 0/1 Completed 0 107s
Retrieving the Pods from the application
In my case, it seems that the first run of the experiment was successful. To be on the safe side, we’ll take a look at the Pods of the demo applications.
kubectl --namespace go-demo-8 get pods
The output is as follows.
NAME READY STATUS RESTARTS AGE
go-demo-8-... 2/2 Running 2 27m
go-demo-8-... 2/2 Running 3 27m
go-demo-8-db-... 2/2 Running 0 27m
repeater-... 2/2
...