Backpropagation Part 1

Backpropagation

Backpropagation involves the steps of updating the Neural Network weights while training the Neural Network on the training dataset, and it updates the weights in such a way that the total error for the problem under consideration is minimized.

What is backward pass

It involves propagating the error (calculated at the output layer) backwards to the layers and updating the weights.

General context of a neuron and formulas

If we look into the general context of a neuron from the diagram above, we get the following equations.

• $a_j=\sum_{i}w_{ij}z_{i}$: This sums up all the inputs multiplied with the respective weights.

• $z_j=\sigma{(a_j)}=\frac{1}{1+e^{-a_j}}$ This applied the activation function on the input being passed in.

• $Error=\frac{1}{2}\sum_{instances}\sum_{k}(y_k-t_k)^2$ ...