Search⌘ K
Home
Courses
Make Your Own Neural Network in Python

Evaluate the Performance of the Neural Network

Explore how to evaluate your neural network's performance by training on 60,000 MNIST images and testing on 10,000 images. Learn to calculate accuracy scores, interpret results, and compare your network's effectiveness to established benchmarks. This lesson helps you understand practical performance testing for your Python-built neural network.

We'll cover the following...

Training and testing with a full dataset

Previously, we added a simple code to work out the fraction of correct answers. Let’s see what this updated code produces:

Python 3.5
# test the neural network
# scorecard for how well the network performs, initially empty
scorecard = []
# go through all the records in the test data set
for record in test_data_list:
    # split the record by the ',' commas
    all_values = record.split(',')
    # correct answer is first value
    correct_label = int(all_values[0])
    print(correct_label, "correct label")
    # scale and shift the inputs
    inputs = (numpy.asfarray(all_values[1:]) / 255.0 * 0.99) + 0.01
    # query the network
    outputs = n.query(inputs)
    # the index of the highest value corresponds to the label
    label = numpy.argmax(outputs)
    print(label, "network's answer")
    # append correct or incorrect to list
    if (label == correct_label):
        # network's answer matches correct answer, add 1 to scorecard
        scorecard.append(1)
    else:
        # network's answer doesn't match correct answer, add 0 to scorecard
        scorecard.append(0)
        pass
    
    pass
  
print(scorecard)
# calculate the performance score, the fraction of correct answers
scorecard_array = numpy.asarray(scorecard)
print ("performance = ", scorecard_array.sum() / scorecard_array.size)
print ("performance percentage = ", (scorecard_array.sum() / scorecard_array.size)*100,"%")

If we get a performance score of 0.7, that means we achieved 70% accuracy.

A sample output
A sample output

Discussion of the results

Let’s add all this new code we’ve just developed to test the neural network’s performance to our main program.

While we’re at it, let’s change the file names so that we are now pointing to the full training data set of 60,000 records, ...