Best + Worst Curves and Models
Understand the best and worst possible curves as well as the comparison between the best and worst models.
We'll cover the following...
Best and worst curves
Let us ask ourselves: what would the best possible (and the worst possible) curves look like?
The best curve belongs to a model that predicts everything perfectly; it gives us a 100% probability to all actual positive data points and 0% probability to all actual negative data points. Of course, such a model does not exist in real life. But cheating does exist. So, let us cheat and use the true labels as the probabilities. These are the curves we get:
Nice! If a perfect model exists, its curves are actually squares! The top-left corner on the ROC curve, as well as the top-right corner on the PR curve, are the (unattainable) sweet spots. Our logistic regression was not bad actually. But, of course, our validation set was straightforward.
...
“And the Oscar for the worst curve goes to…”
“…the random model! ...