Training GANs

Training a GAN is like playing this game of two adversaries. The generator is learning to generate good enough fake samples, while the discriminator is working hard to discriminate between real and fake. More formally, this is termed the minimax game, where the value function $V(G, D)$ is described as follows:

This is also called the zero-sum game, which has an equilibrium that is the same as the Nash equilibrium. We can better understand the value function $V(G, D)$ by separating out the objective function for each of the players. The following equations describe individual objective functions:

Where $J^D$ is the discriminator objective function in the classical sense, $J^G$ is the generator objective equal to the negative of the discriminator, and $p_{data}$ is the distribution of the training data. The rest of the terms have their usual meaning. This is one of the simplest ways of defining the game or corresponding objective functions.

The objective functions help us to understand the aim of each of the players. If we assume both probability densities are non-zero everywhere, we can get the optimal value of $D(x)$ as:

The next step is to present a training algorithm wherein the discriminator and generator models train toward their respective objectives.

Training algorithm

The simplest yet widely used way of training a GAN (and by far the most successful one) is as follows:

Repeat the following steps $N$ times. $N$is the number of total iterations:

• Step 1: Repeat steps $k$ times:

  • Sample a minibatch of size $m$ from the generator: {${z_1,z_2,...z_m}$ ...