Visualizing Training

When we trained networks in Make Your Own Neural Network, we didn’t really have a way of seeing the progress of that training. We did measure how well the network performed after training, but we didn’t get a sense of how smoothly the training itself had gone, or a sense of whether further training would be helpful.

One way to keep track of training is to monitor the loss.

Track the loss value during training

We could do this by keeping a copy of the loss value in a list every time it is calculated inside train(). This would mean the list would get very big because neural network training is often run for thousands, if not millions, of examples. The MNIST dataset has 60,000 training examples, and we might run several epochs of these. A better way is to keep a copy of the loss value after every 10 training examples. This means we need to keep a count of how often train() has been run.

The following code creates a counter, initially set to 0, and an empty list called progress in the constructor of the neural network class.