The Variational Objective

Let’s examine how to optimally compress information in numerical vectors using neural networks. To do so, each element of the vector should encode distinct information from the others, a property we can achieve using a variational objective. This variational objective is the building block for creating VAE networks.

Creating efficient encodings

Let’s start by quantifying more rigorously what makes such an encoding “good” and allows us to recreate images well. We'll need to maximize the posterior: