Overfitting Intuition

The persistent challenge in machine learning

A critical concept in using machine learning effectively is overfitting. Overfitting is where a model’s predictions are much less accurate for data not used in training the model (i.e., less accurate with new data).

Now here’s the thing.

Crafting machine learning models that overfit is ridiculously easy. Much of what machine learning practitioners do to craft the most valuable models (e.g., engineering new features) tends to make the models overfit.

To craft the most valuable models, it is imperative to look for signs that a model tends to overfit constantly.

Later lessons will cover overfitting in-depth. For right now, the goal is to build an intuitive understanding.

The model is good!

Consider the effectiveness of the following model’s predictions. The model was trained with ten observations and correctly predicted the income label for nine out of ten observations. The model is 90 percent accurate on the training data!

However, this isn’t a reason to celebrate quite yet.