Fundamentals of Machine Learning for Software Engineers/

...

/

Launch the Classifier

Launch the Classifier

Apply what we have learned regarding multi-class classifiers and get a bird's eye view of what we have covered in previous chapters.

We'll cover the following...

Multiclass classifier

It’s time to launch our multiclass classifier on MNIST. Here is the classifier’s code, in all its glory:

# An MNIST loader.

import numpy as np
import gzip
import struct


def load_images(filename):
    # Open and unzip the file of images:
    with gzip.open(filename, 'rb') as f:
        # Read the header information into a bunch of variables:
        _ignored, n_images, columns, rows = struct.unpack('>IIII', f.read(16))
        # Read all the pixels into a NumPy array of bytes:
        all_pixels = np.frombuffer(f.read(), dtype=np.uint8)
        # Reshape the pixels into a matrix where each line is an image:
        return all_pixels.reshape(n_images, columns * rows)


def prepend_bias(X):
    # Insert a column of 1s in the position 0 of X.
    # (“axis=1” stands for: “insert a column, not a row”)
    return np.insert(X, 0, 1, axis=1)


# 60000 images, each 785 elements (1 bias + 28 * 28 pixels)
X_train = prepend_bias(load_images("../programming-machine-learning/data/mnist/train-images-idx3-ubyte.gz"))

# 10000 images, each 785 elements, with the same structure as X_train
X_test = prepend_bias(load_images("../programming-machine-learning/data/mnist/t10k-images-idx3-ubyte.gz"))


def load_labels(filename):
    # Open and unzip the file of images:
    with gzip.open(filename, 'rb') as f:
        # Skip the header bytes:
        f.read(8)
        # Read all the labels into a list:
        all_labels = f.read()
        # Reshape the list of labels into a one-column matrix:
        return np.frombuffer(all_labels, dtype=np.uint8).reshape(-1, 1)


def one_hot_encode(Y):
    n_labels = Y.shape[0]
    n_classes = 10
    encoded_Y = np.zeros((n_labels, n_classes))
    for i in range(n_labels):
        label = Y[i]
        encoded_Y[i][label] = 1
    return encoded_Y


# 60K labels, each a single digit from 0 to 9
Y_train_unencoded = load_labels("../programming-machine-learning/data/mnist/train-labels-idx1-ubyte.gz")

# 60K labels, each consisting of 10 one-hot encoded elements
Y_train = one_hot_encode(Y_train_unencoded)

# 10000 labels, each a single digit from 0 to 9
Y_test = load_labels("../programming-machine-learning/data/mnist/t10k-labels-idx1-ubyte.gz")
The MNIST classifier

We also extract a new report() function that logs the percentage of correct results. The report() function is similar to the test() function, but it is called once per training iteration, plus once at the very end. This log shows us in detail how well (or badly) our classifier is learning.

A tiny fix

We would have to be very vigilant to spot it, but the final classifier code introduces a subtle change into the last line of loss(). So far, that line uses numpy’s average() function to calculate the average of (first_term + second_term). However, average() calculates the average over all the elements of a matrix, while we want the loss over the examples—which includes all the lines in the matrix.

So far, that detail didn’t matter because first_term + second_term had exactly one element per line. Since we switch to ...