Introduction to Linear Regression

This lesson will discuss linear regression, which is a straightforward approach and is considered the "workhorse" approach for supervised machine learning. Linear regression has been around for a long time and is the topic of countless textbooks. It's a practical and widely used statistical or machine learning model. Moreover, it serves as a good jumping-off point for newer approaches.

History

The earliest form of regression was the least squares method, which Legendre published in 1805 and was later published by Gauss in 1809. However, the term "regression" was coined by Sir Francis Galton in his work, published in 1875, while he was describing the biological phenomenon of relating the heights of descendants to their tall ancestors. For Sir Galton, regression had only this biological meaning, but Udny Yule and Karl Pearson later extended his work to a more general statistical context.

In his study, Sir Galton discovered that a man's son tends to be roughly as tall as his father, but the son's height tends to be closer (regress or drift toward) to the overall average height.

Let's consider the simplest possible example: