Backpropagation: Part 2

Backpropagation worked example

In this lesson, we will consider the neural network below and update the weights and biases for one round of backpropagation. We use the sigmoid function as the activation function, and the learning rate l is 0.9.

We have the following initial values of inputs, weights, and biases of the above neural network:

$x_1, x_2, x_3$ are our input values with values $1$, $0$ and $1$, respectively, and the corresponding class label is 1 i.e $t_j=1$. $\theta_{4}, \theta_{5}, \theta_{6}$ are the bias values of the units $4, 5$ and $6$, respectively.

Feedforward process

The following table sums up the feedforward process. The first column represents the unit number from the above neural network. The second column represents the net input being computed at those units, and the third column applies the sigmoid activation on the relevant net inputs and produces the output.

Formulas for weights update

From the last lesson, we have the following formulas, and we will be utilizing them to update the weights and biases.

• $w_{ij}^{new} = w_{ij}^{old} + \Delta w_{ij}$ ...