Make Your Own Neural Network in Python/

...

/

Do Multiple Runs

Do Multiple Runs

Explore how the number of epochs affects the accuracy of results.

We'll cover the following...

Epochs

The next improvement we can make is to repeat the training several times against the dataset. This is sometimes called an epoch. So, a training session with 10 epochs means running through the entire training dataset 10 times. Why would we do that, especially it takes 10, 20, or even 30 minutes at a time? It is worth doing because we’re helping those weights do that gradient descent by providing more chances to creep down those slopes.

Let’s try it with two epochs. The code changes slightly because now we add an extra loop around the training code. The following code shows this outer loop to help us see what’s happening:

Press + to interact
# train the neural network
# epochs is the number of times the training data set is used for training
epochs = 2
for e in range(epochs):
    # go through all records in the training data set
    for record in training_data_list:
        # split the record by the ',' commas
        all_values = record.split(',')
        # scale and shift the inputs
        inputs = (numpy.asfarray(all_values[1:]) / 255.0 * 0.99) + 0.01
        # create the target output values (all 0.01, except the desired label which is 0.99)
        targets = numpy.zeros(output_nodes) + 0.01
        # all_values[0] is the target label for this record
        targets[int(all_values[0])] = 0.99
        n.train(inputs, targets)
        pass
    pass

The resulting performance, with two epochs, will be slightly improved as compared to the performance generated over just one epoch (see the figure below).

Performance graph against number of epochs

Just like we did when tweaking the learning rate, let’s experiment with a few different epochs and plot a graph to visualize the effect this has. Our intuition suggests that the more training we do, the better the performance. Some of us will realize that too much training may not be a good idea because the network overfits to the training ...