What is Chroma Subsampling in JPEG?

Introduction#

This blog discusses an important compression technique for colored images known as chroma subsampling. This technique is interesting because it is based on insights about the human visual system. The main idea is to achieve compression by discarding the image data which we can't see or to which we are less sensitive. Compression techniques in which some data or information is lost are known as lossy compression methods.

Digital images can be used for various practical purposes. They may be used for human consumption or for machines. It is not always wise to use chroma subsampling if the image is intended for machines. However, it is a useful method of lossy compression if the digital image is to be intended for the human visual system.

Basics of the human visual system#

To make a decision during lossy compression about losing a part of the information or image data, it is important to understand how our visual system works. When we look at some object, its flipped image is made at the retina through the lens of the eye. Look at the following image showing an illustrated cross-section of the human eye:

At the retina, several tiny photoreceptive sensors sense this image and send the information to the brain through the optic nerve. The photoreceptive sensors at the retina are broadly classified into two types, rods and cones. The rods are very sensitive to light, while the cones are responsible for color vision. The number of rods in the retina of the human eye is approximately 91 million, while the cones are approximately 4.5 million. This means that the human eye is more sensitive to luminance than to color in visual data. This insight is the foundation of the idea that we can reduce the resolution of the color component without compromising the perceived visual quality of the image.

Color space transformation#

Since we want to reduce the resolution of the color component in the image, we first have to separate the color part from the luminance part of the image. This can be achieved by transforming the image from $RGB$ to the $YC_bC_r$ color space. In the $YC_bC_r$ color space, the $Y$ component contains the luminance part, whereas $C_b$ and $C_r$ contain the chrominance part of the image. Once we transform the image to this format, we'll be able to reduce the resolution of the color part of the image. An image can be converted from $RGB$ to the $YC_bC_r$ color space with the help of the following set of equations:

Three components, $R$, $G$, and $B$, of each pixel in the image are used on the right-hand side of these equations to get corresponding $Y$, $C_b$, and $C_r$ values. In this pixel-wise manner, the color space of the whole image is transformed. If we want to convert back to the $RGB$ color space, we can do so by using the following set of equations:

These equations are also used pixel-wise to transform the color space of the image. Both the set of equations discussed above are for the case when the $R$, $G$, and $B$ values of the pixels in the image are in the range of $0$ to $255$. That means one byte of storage is used to represent $R$, $G$, and $B$ each, and one complete pixel is represented in three bytes in this way. If that is not the case, then we have to adjust the coefficients in both the set of equations presented above.

Subsampling chrominance components#

The term subsampling means considering a reduced number of samples from the available samples for the same amount of space. As discussed above, each pixel of an image in $RGB$ consists of three samples, one for each primary color. When we transform the image to $YC_bC_r$, against each pixel, we get three values, namely $Y$, $C_b$, and $C_r$. We can subsample $C_b$ and $C_r$ by keeping only one out of four samples of each with the corresponding four samples of $Y$. In this way, we can represent four pixels by using six values instead of twelve values. This subsampling technique is also referred to as 4:2:0. There can be different methods to come up with one representative value of each $C_b$ and $C_r$ instead of four values for each. The representative sample value can be just one of the four values of the existing samples, or it can be the average of these. The 4:2:0 subsampling process is further explained through the following illustration:

The leftmost column of the illustration shows three layers of sixteen pixels in the $RGB$ format. Each layer shows values of sixteen pixels for one primary color. The small black dot represents the value of a primary color of a single pixel. The middle column of the illustration shows the sixteen pixels after transformation from $RGB$ to the $YC_bC_r$ color space. In the right column of the illustration, the situation after subsampling is shown. The $Y$ component remains the same, while each $C_b$ and $C_r$ now contains only four values. Each value of $C_b$ and $C_r$ in the right column is shown in the center of the place of the four values that it has replaced. The dotted lines show the connection between the new value with the four values which were there before subsampling.

There are other subsampling methods like 4:2:2, in which along four samples of $Y$, two samples of each $C_b$ and $C_r$ are taken. Hence, four pixels that need twelve bytes in $RGB$ can be represented by eight bytes in the 4:2:2 format.

How much compression is achieved#

Let's calculate the amount of space saved by using chroma subsampling. Take an image of full HD size, that is $1920 \times 1080$ pixels. Assume each pixel is using three bytes, one for each primary color. We can compute its size as follows:

The transformation from $RGB$ to $YC_bC_r$ does not change the size of the image. After subsampling, the resolution of the $Y$ component remains the same; however, the size of each $C_b$ and $C_r$ is reduced to a quarter in the case of 4:2:0 format because we are using only one sample to represent four. The size of the image after subsampling can be computed as follows:

The compression ratio is defined as the ratio of the uncompressed size to the compressed size. The amount of space saved, expressed as a fraction of the uncompressed size, can be computed as follows:

Let's compute these quantities for the example we are working with:

This shows that by using 4:2:0 chroma subsampling, we can save half of the space required to store or transmit an image. In the case of 4:2:2, the compression ratio is $1.5$, and space saved is $\frac{1}{3}$. That means representing an image in 4:2:2 format can save one-third of the required space for storage or transmission.

Chroma subsampling in JPEG#

The famous image compression standard JPEG uses 4:2:0 chroma subsampling as part of its algorithm while compressing colored images in lossy mode. JPEG also supports other subsampling formats like 4:4:4, 4:2:2, and 4:1:1. In the 4:1:1 format, the chroma component is one-fourth, taking one sample per four samples in a row. In 4:4:4, there is no subsampling involved, and the chroma component is completely preserved. The JPEG standard recommends a collection of algorithms applied one after the other to compress a given image, and chroma subsampling is one of those algorithms. The 4:2:0 chroma subsampling is also part of many other image and video compression standards, including MPEG and H.26x.

To dive a little deeper and have experience implementing the ideas presented in this blog, you may like to visit the following hands-on projects:

• Image Compression Through Subsampling and Interpolation

• Image Compression Using DCT with Graceful Quality Degradation