Introduction to Regularization

Let's start with understanding the difference between underfitting and overfitting.

Underfitting vs. overfitting

Let's say we have the following plots (shown below):

• Left plot: Underfitted model with low accuracy score (R-squared) and higher error (SSE: sum of squared error).

• Middle plot: Model with average accuracy score and error. Can this be improved?

• Right plot: The overfitted model has a very high accuracy score and low error (practically R-squared = 1 and SSE = 0 since the fitted line passes through all the data points).