The Echidna Dataset

Deep learning

• Deep learning is a set of complex technologies, but they all spring from a simple idea: neural networks become more powerful if they have many layers. Those “deep” networks pose more challenges than the “shallow” three-layered networks we’ve built so far. In the next few chapters, we’ll describe and overcome those challenges.

• Adding layers is just the beginning. In the last few years, the original concept of deep learning branched out into many innovative and sometimes wonderful ideas. We’ll take a look at some of those.

• In Parts I and II, we used MNIST as our benchmark. MNIST will also be our starting point in the first chapters of Part III, but eventually, we’ll outgrow it and tackle more complex datasets.

• The third part of this course is all about deep learning, the major breakthrough in modern artificial intelligence. Deep learning means a few different things, but first and foremost as mentioned previously, it stands for neural networks with many layers. Deeper networks have more sophisticated models than shallow networks, so they can generate more complicated functions to approximate data.

Later on, we’ll see that there is more to deep learning than adding layers. Deep learning is actually a set of interconnected techniques with names such as convolutions, recurrent neural networks, and generational adversarial networks. By the end of this course, we’ll have a better idea of those techniques, and will be well equipped to explore them on our own.

To begin, we’ll start from that basic concept: adding layers to a neural network. In this chapter, we’ll create two networks:

1. A shallow one with three layers
2. A slightly deeper one with four

We’ll run both networks on the same dataset and compare their results.

In the first two parts of this course, we wrote our code from scratch, line by line. However, as we move into deep learning, this approach takes up more and more time. It also becomes less compelling because we already have a solid grasp of the fundamentals, and do not need to linger on every little detail. From this chapter onward, we’ll write our neural networks with Keras, a popular ML library. We’ll focus on the big picture and Keras will take care of the details.

Let’s recap what we can expect from this chapter:

• We’ll see how to build a neural network with Keras.
• We‘ll write a three-layered network and run it on a simple dataset.
• We’ll add a layer to the network and see how its performance changes.

Prepare Echidna dataset

Before we get into building networks, let’s prepare a dataset to run them on. Let’s recall the concept of Tracing a Boundary. Soon enough, we’ll see how a neural network’s decision boundary changes as we add a fourth layer. For that purpose, there is a dataset that’s twisty, but also easy to visualize. Here it is: