Generative Adversarial Networks (GANs)

Generative adversarial networks (GANs) have a pretty interesting origin story. It all began as a discussion/argument in a bar with Ian Goodfellow and friends discussing work related to generating data using neural networks. The argument ended with everyone downplaying each other’s methods. Goodfellow went back home and coded the first version of what we now call a GAN. To his amazement, the code worked on the first try. A more verbose description of the chain of events was shared by Goodfellow himself in an interview with Wired magazine.

Taxonomy of generative models

The first set of methods corresponds to models that represent data with an explicit density function. Here, we define a probability density function, $p\theta$, explicitly and develop a model that increases the maximum likelihood of sampling from this distribution.

There are two further types within explicit density methods: tractable and approximate density methods. PixelRNNs are an active area of research for tractable density methods. When we try to model complex real-world data distributions, for example, natural images or speech signals, defining a parametric function becomes challenging. These techniques work by approximating the underlying probability density functions explicitly. VAEs work toward maximizing the likelihood estimates of the lower bound, while RBMs use Markov chains to make an estimate of the distribution. The overall landscape of generative models can be described using the figure belowGoodfellow, Ian. 2016. “NIPS 2016 Tutorial: Generative Adversarial Networks.” ArXiv.org. 2016. https://arxiv.org/abs/1701.00160..

GANs fall under implicit density modeling methods. The implicit density functions give up the property of explicitly defining the underlying distribution but work by defining methods to draw samples from such distributions. The GAN framework is a class of methods that can sample directly from the underlying distributions. This alleviates some of the complexities associated with the methods we have covered so far, such as defining underlying probability distribution functions and the quality of outputs.

The GAN framework

GANs are implicit density functions that sample directly from the underlying distribution. They do this by defining a ...