Building The Generator

Building the generator needs a little more thought, so we’ll proceed step by step.

The generator is a neural network, and not a simple function because we want it to learn. We hope its output will get past the discriminator. That means the last output layer needs 4 nodes, to match the real data.

How big should the hidden layer be? What about the generator’s input layer? There isn’t anything that forces us to use a specific size for these. They should be large enough to learn, but not so big that it takes a very long time, and with an overall aim to match the rate at which the discriminator learns so one doesn’t get too far ahead of the other. For these reasons, many researchers start by mirroring the discriminator.

Generator architecture

So let’s try a generator with an input layer of 1 node, a hidden layer with 3 nodes, and an output layer of 4 nodes, which is the discriminator in reverse.