Summary: Painting Pictures with Neural Networks Using VAEs

In this chapter, we learned how deep neural networks can be used to create representations of complex data, such as images, that capture more of their variance than traditional dimension reduction techniques, such as PCA. This is demonstrated using the MNIST digits, where a neural network can spatially separate the different digits in a two-dimensional grid more cleanly than the principal components of those images.

The chapter explained how deep neural networks can be used to approximate complex posterior distributions, such as images. Using variational methods to sample from an approximation of an intractable distribution, the chapter led to a VAE algorithm based on minimizing the variational lower bound between the true and approximate posterior.

We also learned how the latent vector from this algorithm can be reparameterized to have lower variance, leading to better convergence in stochastic minibatch gradient descent. We saw how the latent vectors generated by encoders in these models, which are usually independent, can be transformed into more realistic correlated distributions using IAF. Finally, we implemented these models on the CIFAR-10 dataset and showed how they can be used to reconstruct the images and generate new images from random vectors.