Basics of Autoencoders

Before we discuss variational autoencoders, let’s first see how the standard autoencoders work.

Autoencoders are simple neural networks such that their output is their input.

It is as simple as that.

Their goal is to learn how to reconstruct the input data.

But how is it possible? The trick is their structure.

The first part of the network is what we refer to as the encoder. It receives the input and encodes it in a latent space of a lower dimension (the latent variables $z$).

For now, you can think of the latent space as a continuous low-dimensional space.

The second part (the decoder) takes that vector and decodes it in order to produce the original input.

The latent vector $z$ ...