Rescale the Target Output
Learn the importance of rescaling the input and output values to achieve good results.
We'll cover the following
The need to rescale the output
We now need to think about the neural network’s outputs. We saw earlier that the outputs should match the range of values that the activation function can push out. The logistic function we’re using can’t push out numbers like or . The range is , and in fact, we can’t reach or because the logistic function can only approach these extremes and will never actually get there. So, it looks like we’ll have to scale our target values when training.
What should the output even be? Should it be an image of the answer? That would mean we’d have output nodes.
If we take a step back and think about what we’re asking the neural network to do, we realize we’re asking it to classify the image and assign the correct label. That label is one of numbers, or . That means the network should be able to have an output layer of nodes, one for each of the possible answers, or labels. If the answer is , the first output layer node would fire and the rest should be silent. If the answer is , the last output layer node would fire and the rest would be silent.
Examples
The following table illustrates this scheme with some example outputs:
Get hands-on with 1400+ tech skills courses.