Feed Forward Propagation

We know what is forward propagation while calculating the predicted value for the perceptron algorithm.

Feedforward in a neural network

Feedforward is the process that is used by neural networks to turn the given input into an output.

We have learned that the neural network is a combination of perceptrons. The perceptrons in each layer compute the output, apply the activation function, and will pass the information to the next layer. This process continues until the output layer is reached. In short, each node feeds the information it has to the next node. For this reason, it is called a feed-forward propagation algorithm.

Visualize feedforward propagation

The following illustration shows a feedforward propagation algorithm using a neural network with a hidden layer comprising of two hidden units as seen below:

The above illustration shows forward propagation for a single data point. In general, we should do the forward pass for every data point. The value in the last layer is the model’s prediction for that data point.

📝Note: By using the perceptron equation, it computes the net output.

📝Note: By applying the activation function to the net output, it computes the activated output.

The activation function used in the illustration above is $sigmoid$ (σ).

The feedforward operation is a multiply and add process which is a dot product in vector algebra.

Visualize this in the illustration below:

📝 Note: In the above illustration, bias is added in the weight’s vector array. However, we can create a separate bias array. ...