A Quick Recap of Common Metrics

Let’s review basic model performance metrics.

We'll cover the following

Before proceeding with fairness-specific metrics, it is essential to recap the basic ones because they are usually the foundation.

The first step is to define which class we consider positive. For analyzing fairness, it is often more convenient to assign the positive class to a desirable outcome for the user. However, sometimes it is set to an unwelcome prediction. If a model is used to decide if a suspect should be temporarily arrested (positive class) or released (negative class), they would like to be assigned to the negative one. Interpretation of metrics changes when we reverse labels, so we need to be careful.

Confusion matrix

One important way of presenting classifier results is the confusion matrix (CM). It is a table that represents the performance of the model. Specific metrics can be derived directly from values in the confusion matrix, making it a great starting point.

Getting back to Space UniversitySpaceUni, a model evaluation could give us the following matrix:

Get hands-on with 1200+ tech skills courses.