Transform the Output into an Error Function

Error functions

The output of a neural network is a complex difficult function with many parameters and the link weights that influence its output. Can we use gradient descent to work out the right weights? Yes, as long as we pick the right error function.

The output function of a neural network isn’t an error function itself. But we know we can turn it into one easily, because the error is the difference between the target training values and the actual output values.

There’s something to watch out for here. Look at the following table of training and actual values for three output nodes, together with candidates for an error function.