The Best-Fitting Line

Regression lines are also known as best-fitting lines. What do we mean by best? Let’s explore the criteria that’s used in regression to determine the concept of best. Recall that, for an instructor with a beauty score of$𝑥 = 7.333$, we mark the observed value $y$ with a circle, the fitted value $\hat y$ with a square, and the residual $y - \hat y$ with an arrow. We redisplay the top left plot of the figure below in addition to three more arbitrarily chosen course instructors: