Overfitting and underfitting

Overfitting and underfitting are two of the most common causes of poor model accuracy. The model fit can be predicted by taking a look at the prediction error on the training and test data.

An underfit model results in high prediction errors for both training and test data. An overfit model gives a very low prediction error on training data, but a very high prediction error on test data. Both types of models result in poor accuracy.

An underfit model fails to significantly grasp the relationship between the input values and target variables. This may be the case when the model is too simple (i.e., the input features are not explanatory enough to describe the target well).

An overfit model has overly memorized the data set it has seen and is unable to generalize the learning to an unseen data set. That is why an overfit model results in very poor test accuracy. Poor test accuracy may occur when the model is highly complex, i.e., the input feature combinations are in a large number and affect the model’s flexibility.