Text-to-Image Synthesis with GANs

In this chapter, we will gradually move from CV to NLP by combining the two fields together, and we will try to generate realistic images from the description text. This process is called text-to-image synthesis (or text-to-image translation).

We know that almost every GAN model generates synthesized data by establishing a definite mapping from a certain form of input data to the output data. Therefore, in order to generate an image from a corresponding description sentence, we need to understand how to represent sentences with vectors.

Quick introduction to word embedding

It is rather easy to define an approach for transforming the words in a sentence into vectors. We can simply assign different values to all the possible words (for example, let 001 represent I, 002 represent eat, and 003 represent apple) so that the sentence can be ...