Measuring ML Model Performance
Learn how to measure machine learning model performance. This lesson discusses training error, testing error, overfitting, underfitting, bias, variance, and dataset splitting.
We'll cover the following...
When we build the ML model, we use the data and create the model. Consider the problem of predicting the skill level of a student from his answers to a different question. We take the data and create a classification model that tells us the skill of a student on a scale of 1 to 5 (integers). We verify this model on the data available to build the model.
However, what happens when we deploy this model to production? Maybe it is good for some students, but it gives bad results for others. Maybe some highly skilled students got lower ratings due to wrong answers to a particular question. So, we may end up either creating a good model or a bad model that no one wants to use. Our goal is to always create the best model. In cases where business depends on ML models, it is necessary to create a reliable, good model.
Loss function
So, we want to know how successful the model is or the amount of loss associated with our predictions. This can be measured with a loss function.
Loss function = L(y,fw(x))
Here, y stands for the true value, and fw(x) is the predicted value. L ...