Delve into advanced Kubernetes features like monitoring, alerting, logging, and auto-scaling. Gain insights into creating resilient, self-adaptive clusters and applications with minimal manual intervention.

The world of DevOps has exploded with the help of technologies like Kubernetes and Docker.  
In this course, you will learn about more advanced Kubernetes features such as monitoring, alerting, logging, and scaling - crucial concepts for any DevOps professional. You will learn how to monitor clusters, how to send alerts when potential issues arise, and how to query the whole system’s metrics and logs. 
By the end of this course, you will have the required knowledge to make your clusters and applications truly dynamic and resilient, so that they require minimal manual involvement and are self-adaptive.

Advanced Kubernetes Techniques: Monitoring, Logging, Auto-Scaling

`HorizontalPodAutoscaler (HPA)` and `Cluster Autoscaler (CA)` provide essential, yet very rudimentary mechanisms to scale our Pods and clusters. While they do scaling decently well, they do not solve our need to be alerted when there's something wrong, nor do they provide enough information required to find the cause of an issue. We'll need to expand our setup with additional tools that will allow us to store and query metrics as well as to receive notifications when there is an issue.

## Store and query metrics using `Prometheus`

If we focus on tools that we can install and manage ourselves, there is very little doubt about what to use. If we look at the list of [Cloud Native Computing Foundation (CNCF) projects](https://www.cncf.io/projects/), only two graduated in October 2018. Those are **Kubernetes** and [Prometheus](https://prometheus.io/). Given that we are looking for a tool that will allow us to store and query metrics and that `Prometheus` fulfills that need, the choice is straightforward. That is not to say that there are no other similar tools worth considering. There are, but they are all service-based. We might explore them later but, for now, we're focused on those that we can run inside our cluster. So, we'll add `Prometheus` to the mix.

### What is `Prometheus`?

> 📌 <code>Prometheus</code> is a database (of sorts) designed to fetch (pull) and store highly dimensional time series data.

Time series are identified by a metric name and a set of key-value pairs. Data is stored both in memory and on disk. Former allows fast retrieval of information, while the latter exists for fault tolerance.

### How `Prometheus`'s works?

`Prometheus'` query language allows us to easily find data that can be used both for graphs and, more importantly, for alerting. It does not attempt to provide a "great" visualization experience. For that, it integrates with [Grafana](https://grafana.com/).

Unlike most other similar tools, we do not push data to `Prometheus`. Or, to be more precise, that is not the common way of getting metrics. Instead, `Prometheus` is a pull-based system that periodically fetches metrics from exporters. There are many third-party exporters we can use. But, in our case, the most crucial exporter is baked into **Kubernetes**. `Prometheus` can pull data from an exporter that transforms information from Kube API. Through it, we can fetch (almost) everything we might need. Or, at least, that's where the bulk of the information will be coming from.

## Send alerts using `AlertManager`

Finally, storing metrics in `Prometheus` would not be of much use if we are not notified when there's something wrong. Even when we do integrate `Prometheus` with [Grafana](https://grafana.com/), that will only provide us with dashboards. I assume that you have better things to do than to stare at colorful graphs. So, we'll need a way to send alerts from `Prometheus` to, let's say, Slack. Luckily, [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) allows us just that. It is a separate application maintained by the same community.

We'll see how all those pieces fit together through hands-on exercises. So, let's get going and install `Prometheus`, `Alertmanager`, and a few other applications.


`HorizontalPodAutoscaler (HPA)` and `Cluster Autoscaler (CA)` provide essential, yet very rudimentary mechanisms to scale our Pods and clusters. While they do scaling decently well, they do not solve our need to be alerted when there's something wrong, nor do they provide enough information required to find the cause of an issue. We'll need to expand our setup with additional tools that will allow us to store and query metrics as well as to receive notifications when there is an issue.

# Store and query metrics using `Prometheus`

If we focus on tools that we can install and manage ourselves, there is very little doubt about what to use. If we look at the list of [Cloud Native Computing Foundation (CNCF) projects](https://www.cncf.io/projects/), only two graduated in October 2018. Those are **Kubernetes** and [Prometheus](https://prometheus.io/). Given that we are looking for a tool that will allow us to store and query metrics and that `Prometheus` fulfills that need, the choice is straightforward. That is not to say that there are no other similar tools worth considering. There are, but they are all service-based. We might explore them later but, for now, we're focused on those that we can run inside our cluster. So, we'll add `Prometheus` to the mix.

## What is `Prometheus`?

> 📌 <code>Prometheus</code> is a database (of sorts) designed to fetch (pull) and store highly dimensional time series data.

Time series are identified by a metric name and a set of key-value pairs. Data is stored both in memory and on disk. Former allows fast retrieval of information, while the latter exists for fault tolerance.

## How `Prometheus`'s works?

`Prometheus'` query language allows us to easily find data that can be used both for graphs and, more importantly, for alerting. It does not attempt to provide a "great" visualization experience. For that, it integrates with [Grafana](https://grafana.com/).

Unlike most other similar tools, we do not push data to `Prometheus`. Or, to be more precise, that is not the common way of getting metrics. Instead, `Prometheus` is a pull-based system that periodically fetches metrics from exporters. There are many third-party exporters we can use. But, in our case, the most crucial exporter is baked into **Kubernetes**. `Prometheus` can pull data from an exporter that transforms information from Kube API. Through it, we can fetch (almost) everything we might need. Or, at least, that's where the bulk of the information will be coming from.

# Send alerts using `AlertManager`

Finally, storing metrics in `Prometheus` would not be of much use if we are not notified when there's something wrong. Even when we do integrate `Prometheus` with [Grafana](https://grafana.com/), that will only provide us with dashboards. I assume that you have better things to do than to stare at colorful graphs. So, we'll need a way to send alerts from `Prometheus` to, let's say, Slack. Luckily, [Alertmanager](https://prometheus.io/docs/alerting/alertmanager/) allows us just that. It is a separate application maintained by the same community.

We'll see how all those pieces fit together through hands-on exercises. So, let's get going and install `Prometheus`, `Alertmanager`, and a few other applications.


This lesson focuses on giving a short Introduction to Prometheus and how it can help us to store and query the Metrics, along with sending alerts.

Before Getting Started

Autoscaling Deployments and StatefulSets

Auto-Scaling Nodes Of A Kubernetes Cluster

Collecting and Querying Metrics and Sending Alerts

Debugging Issues Discovered Through Metrics and Alerts

Extending HorizontalPodAutoscaler With Custom Metrics

Visualizing Metrics And Alerts

Collecting And Querying Logs

Conclusion

Choose the Tools for Storing and Querying Metrics and Alerting

Store and query metrics using `Prometheus` #

Choose the Tools for Storing and Querying Metrics and Alerting

Store and query metrics using Prometheus #

Store and query metrics using `Prometheus` #