Unmask the Hypocrite Classifier

The predict_death classifier does not add any insight, it outperforms the random classifier concerning overall accuracy. This exploits the prevalence, the ratio between the two possible values, not being 0.5.

The confusion matrix reveals more details on certain areas. For example, it shows that the predict_death classifier lacks any recall and predicts actual positives. This is no surprise since it always predicts death.

But having a whole set of metrics makes it difficult to measure real progress. How do we recognize that one classifier is better than another? How do we even identify a classifier that adds no value at all? How do we identify such a hypocrite classifier?

Let’s write a generalized hypocrite classifier and see how we can unmask it.