What Are Diffusion Models?

Overview

Consider the scenario of dispersing an ink drop into a container of water. Initially, the drop remains easily distinguishable. However, if given ample time, the molecules will progressively disperse uniformly throughout the container, resulting in a noticeable alteration of the water’s color. This is a fundamental illustration of the principles governing diffusion. Now, let’s explore how the concept of diffusion serves as a source of inspiration for diffusion models.

Diffusion models

Let’s assume we want to generate images, and the water container represents one image. We’ll also assume that the ink drop is noise, which we gradually add to the image. What will happen at each step? We’ll keep adding the noise to the image till the image becomes pure noise. The idea is to learn the distribution of images by gradually undoing the addition of noise.

Components of a diffusion model

Let’s discuss the two parts of a diffusion model.

Forward process

The forward process in a diffusion model starts with a sample taken from input data. We apply the diffusion transformations, such as adding noise iteratively till the data becomes pure noise, to this sample. We usually consider Gaussian noise (normal distribution) in diffusion models. Consider an image $x_o$taken randomly from our data. We add noise $\varepsilon \sim \mathcal{N}(0,I)$ (sampled from normal distribution with mean 0 and varience 1) to $x_o$ in $T$ steps, which will result in a sequence of noisy samples $x_1, x_2, \dots, x_T$.

We model the relationship between $x_t$ and $x_{t-1}$ as the Gaussian distribution, where $q(x_0)$ is the real data distribution:

$q(x_t|x_{t-1}) \coloneqq \mathcal{N}(x_t ; \sqrt{1-\beta_t } \space x_{t-1}, \beta_t I)$ ...