Optimisation Function
Learn about different types of optimisation functions: what they are, their benefits and drawbacks?
We'll cover the following...
We can also refine the technique that uses the back propagated gradients to update the network weights.
Stochastic gradient descent
Previously we used a fairly simple method, stochastic gradient descent. The method is popular because it is simple and fairly lightweight in terms of computer resources.
A weakness with the simple stochastic gradient descent is that it can get stuck in local minima in the ...
Stochastic gradient descent gets stuck in global minima