Model Performance Metrics

Evaluation metrics quantify a machine-learning model’s performance. We improve the model to achieve the desired performance based on these values.

Multiple metrics can be used to evaluate a machine-learning model. However, it is important to know which evaluation metric to use for a given model. Not choosing the right evaluation metric will lead to wrong assumptions about the model’s capability. Let’s understand these metrics:

Performance metrics for classification

The performance metric for classification quantifies the percentage of the objects being classified correctly. Some common performance metrics for classification algorithms are accuracy, precision, and confusion matrix.

Classification algorithms assign a label or category to a given input based on its attributes. Let’s consider an example of a binary classifier that identifies the presence of a disease based on a medical test. In the ML pipeline, we use labeled data to evaluate the performance of the ML model. The model’s predictions are compared to the actual outcomes to calculate the following four attributes:

• True positives (TP): Number of instances where the test correctly identifies the presence of disease.

• False positives (FP): Number of instances where the test incorrectly identifies a healthy person as having the disease.

• True negatives (TN): The number of instances where the test correctly identifies the absence of disease in a healthy person.

• False negatives (FN): Number of instances where the test declares a sick person as healthy.

These four terms make a confusion matrix. The confusion matrix gives the detailed tabular information of a binary classifier. It created a 2x2 matrix for the true and false positives and true and false negatives. The illustration below shows a confusion matrix for a binary classifier:

Accuracy

Accuracy is the ratio of the correct number of predictions to the total predictions. It is formulated as:

This is the simplest type of ...