Activation Functions

A neuron calculates the “weighted sum” of its inputs and adds a bias.

Now, the predicted value can be anything from -∞ to +∞. The neuron does not know the bounds of the value. So, how do we decide whether the neuron should fire or not? Predict whether the class label is positive or negative (in this case, we have two classes). Thus, add an activation function to make the neural network decide whether a neuron should be activated or not. Hence the given name, “activation functions”.