Introduction

Intended audience

This course is for anyone who wants to understand neural networks and develop their own. Furthermore, it’s for anyone who wants to appreciate the fairly easy but exciting mathematical ideas that are at the core of how neural networks work.

The purpose of this course is to explain complex ideas and concepts in a way that a moderately curious school student can understand.

Interested learners and students will build a foundation to go on other exciting excursions into artificial intelligence. Once we’ve grasped the basics of neural networks, we can apply the core ideas to many types of problems.

Teachers can use it as a simple explanation of neural networks and to enthuse and excite students by helping them make their very own learning artificial intelligence with only a few lines of programming language code. The code has been tested on a Raspberry Pi—a small, inexpensive computer that is popular in schools and with young students.

What will we do?

Our end goal is to make a neural network that can recognize human handwritten numbers.

We’ll start with predicting neurons, and we’ll gradually improve on them as we hit their limits. Along the way, we’ll learn about a few mathematical concepts that are needed to understand how neural networks learn and predict solutions to problems.

Next, we’ll look at mathematical concepts such as functions, simple linear classifiers, iterative refinement, matrix multiplication, gradient calculus, optimization through gradient descent, and geometric rotations.

Once we have successfully made our first neural network, we’ll take an idea and experiment with it. For example, we’ll use image processing to improve our machine learning without resorting to additional training data. We’ll even peek inside the mind of a neural network to see if it reveals anything insightful—something not many learning resources show you how to do.

We’ll also learn Python—an easy, useful, and popular programming language—as we make our own neural network in gradual steps.

How will we do it?

Our primary goal is to open up the concepts behind neural networks to as many people as possible. This means that we’ll always start somewhere really comfortable and familiar when explaining an idea. Then we’ll take small, easy steps, building up to where we have enough understanding to appreciate something really cool or exciting about the neural networks.

To keep things as accessible as possible, we’ll resist the temptation to discuss anything that isn’t strictly required to make your own neural network. There will be interesting context and tangents that some learners will appreciate, and if this is you, you’re encouraged to research them more deeply.

We won’t look at all the possible optimizations and refinements to neural networks. There are many options, but they would be a distraction from our core purpose here, which is to introduce the essential ideas in as easy and uncluttered way as possible.

Outline

The outline is intentionally split into three sections:

• The first four chapters of this course are about mathematical ideas at work inside simple neural networks. We’ll focus on the core ideas and won’t introduce any computer programming to avoid getting distracted.

• Chapters 5 and 6 contain an introduction to Python so we can implement our own neural network. We’ll train it to recognize human handwritten numbers and test its performance.

• In chapters 7 through 9, we’ll go further than is necessary to understand simple neural networks, just to have some fun. We’ll try out some ideas to further improve our neural network’s performance. We’ll also have a look inside a trained network to see if we can understand what it has learned and how it decides on its answers.