Creating Separable Encodings of Images

Let’s examine how neural networks can optimally compress information into numerical vectors. To do so, each element of the vector should encode distinct information from the others, a property we can achieve using a variational objective. This variational objective is the building block for creating VAE networks.

In the figure below, you can see an example of images from the CIFAR-10 datasetKrizhevsky, Alex. 2009. “CIFAR-10 and CIFAR-100 Datasets.” Toronto.edu. 2009. https://www.cs.toronto.edu/~kriz/cifar.html., along with an example of an early VAE algorithm that can generate fuzzy versions of these images based on a random number inputPiyush Malhotra, “Variational Autoencoders”, Autoencoder-Implementations. GitHub repository. 2018, https://www.piyushmalhotra.in/Autoencoder-Implementations/VAE/. More recent work on VAE networks has allowed these models to generate much better images.