The Nesterov Momentum

Learn how the Nesterov momentum can be used to escape local optimum in non-convex optimization.

Need for momentum

As shown in the figure below, imagine a ball falling down a valley. If its momentum (mass ×\times velocity) is large enough at the bottom of the valley, there are chances that the ball can escape it and find a deeper valley. However, if the momentum is small, the ball will just move to and fro within the same valley until it settles down at its bottom.

Get hands-on with 1200+ tech skills courses.