Simple Neural Network

Before we dive into writing code for a neural network, let’s step back and draw a picture of what we’re trying to achieve.

A simple network

The following picture shows our starting point and where we want to end up.

The start is an MNIST image which is 28 by 28, or 784 pixel values. That means the first layer of our neural network must have 784 nodes. We don’t really have much of a choice about the size of the input layer.

We do have some choice about the last output layer. It needs to be able to answer the question, “Which digit is this?”. The answer could be any one of the digits from 0 to 9, which is 10 different possibilities. The simplest approach is to have one node for each of these 10 possible classes.

We do have more choice about the hidden middle layer. Because we’re focusing on learning about PyTorch and not trying to find the best size, we’ll pick a size of 200 to get us going.

All the nodes in one layer are connected to every node in the next layer. These are sometimes called fully connected layers.

That picture is missing one key thing. We need to choose an activation function that is applied to the outputs from hidden and output layers.

A simple network with an activation function

In Make Your Own Neural Network we used the s-shaped logistic function. Let’s go with that for simplicity.

Neural network architecture into PyTorch code

We’re ready to turn that neural network architecture into PyTorch code. PyTorch simplifies the task of building and running neural networks by doing a lot of the work behind the scenes. For this to work, we need to follow PyTorch’s coding patterns.

Whenever we create a neural network class we need to inherit it from PyTorch’s own torch.nn. This brings with it ...