Overview: Synthesizing and Manipulating Images with GANs

In this chapter, we will learn how to implement a model based on pix2pixHD, a method for high-resolution (for example, $2048 \times1024$), photorealistic, image-to-image translation. The pix2pixHD model can be used for many exciting tasks, such as turning semantic label maps into photorealistic images or synthesizing portraits from face label maps.

After a brief introduction to the topic of image-to-image translation, we are going to implement a baseline model using the pix2pix setup and train it on the Zappos dataset, which contains images of shoes on a white background and their respective outlines.

After implementing the pix2pix baseline, we will implement a model based on pix2pixHD and train it on the Cityscapes dataset, which focuses on the semantic understanding of urban street scenes and includes images of such scenes and their respective semantic label maps.

Topics covered in this chapter

The following topics will be covered in this chapter:

• Image-to-image translation

• Experimental setup

• Implementation of pix2pix

• Implementation of pix2pixHD