Step 2a - Compute the Loss
Learn about the difference between error and loss, the relationship between gradient descent and loss, and how the loss can be computed.
We'll cover the following...
Difference between error and loss
There is a subtle but fundamental difference between error and loss.
The error is the difference between the actual value (label) and the predicted value computed for a single data point. So, for a given ith point (from our dataset of N points), its error is:
The error of the first point in our dataset (i = 0) can be represented like this:
On the other hand, the loss is some sort of aggregation of errors for a set of data points.
It seems rather obvious to compute the loss for all (N) data points, right? Well, ...