Activation Functions
Learn about different activation functions.
A neuron calculates the “weighted sum” of its inputs and adds a bias.
Now, the predicted value can be anything from -∞ to +∞. The neuron does not know the bounds of the value. So, how do we decide whether the neuron should fire or not? Predict whether the class label is positive or negative (in this case, we have two classes). Thus, add an activation function to make the neural network decide whether a neuron should be activated or not. Hence the given name, “activation functions”.
Types of activation functions
The following section discusses several different types of activation functions that are used in deep learning:
Step function
The first thing that comes to mind is: How about a threshold-based activation function? We saw this activation function when we learned about perceptrons here.
If the predicted value ( ...