A Three-Layer Example: Working on the Input Layer

Three-layered neural network

We haven’t worked through feeding signals through a neural network using matrices to do the calculations. We also haven’t worked through an example with more than two layers, which is relevant because we need to see how we treat the outputs of the middle layer as inputs to the final third layer.

The following diagram shows an example neural network with three layers, each with three nodes. To keep the diagram clear, not all the weights are marked.

Let’s introduce some commonly used terminology here too:

• The first layer is the input layer.
• The middle layer is called the hidden layer.
• The final layer is the output layer.

The name for the middle layer stuck because its outputs are not necessarily made apparent as outputs, so they’re hidden. Let’s work through the example network illustrated in the diagram above. We can see the three inputs are $0.9$, $0.1$, and $0.8$. So the input matrix $I$ is:

$$I= \begin{bmatrix} 0.9\\0.1\\0.8 \end{bmatrix}$$

That was easy. The the first input layer is done because that’s all the input layer does—it merely represents the input. Next is the hidden middle layer.

Calculate the input to the hidden layer

Here, we need to work out the combined (and moderated) signals to each node in the middle layer. Remember, each node in this middle hidden layer is connected to every node in the input layer, so it gets some portion of each input signal. We don’t want to go through all the calculations that we did earlier, so let’s try this matrix method instead.

As we just saw, the combined and moderated inputs into this middle layer are $X = W \cdot I$, where $I$ is the matrix of input signals and $W$ is the matrix of weights. Now we have $I$, but what is ...