Activation Functions

Learn about the most popular activation functions for deep learning.

We'll cover the following...

Activation functions are non-linear functions that determine the outputs of neurons. As we already discussed, each neuron accepts a set of inputs, multiplies them by the weights, sums them up, and adds a bias.

z=w1x1+w2x2+w3x+boz= w_1*x_1 +w_2*x_2 + w_3*x+ bo

Because that will result in a linear transformation, we then pass neurons through a non-linear function ff so we can capture non-linear patterns between our data.

a=f(w1x1+w2x2+w3x+3+bo)a= f( w_1*x_1 +w_2*x_2 + w_3*x+3 + bo) ...