Fine-Tuning Models

Learn fine-tuning techniques to improve a model’s performance.

Underfitting & overfitting

Underfitting refers to the model’s failure to learn. Somehow, the model can’t improve itself, and we see that training and validation accuracy get stuck without getting any better with further iterations. A primary reason for this is to have vanishing or exploding gradients. As we mentioned, both problems are about having bad values to update weights, which causes the learning to stop.

Note: The figures below show us what happens if we set LEARNING_RATE = 1.0.

Press + to interact
Loss and accuracy with LEARNING_RATE = 1.0
Loss and accuracy with LEARNING_RATE = 1.0

The graphs seem awful, and the model can’t learn anything. When we check the gradients, they shrink significantly to 10–28. It’s because even though we select an excellent activation function to fight the vanishing gradient problem, we multiply our gradients with a considerable value, which makes a significant impact when updating our weights and kills them (making them near zero). So we try to carry the input signal until the output layer gets closer to 0. ...

Access this course and 1400+ top-rated courses and projects.