Data Handling

Cross-validation

The performance of a model on the training data can always be improved, and even made perfect, on the training data when making the model more complex. This is the essence of overfitting. Basically, we can always write a model that can memorize a finite dataset. However, machine learning is about generalization, which can only be measured with data points that have not been used during training. This is why in the examples earlier, we split our data into a training set and a test set.