Creating Videos from Images

GANs can generate novel photorealistic images after being trained on a group of example photos. This technique can also be used to create variations of an image, either by applying “filters” or new poses or angles of the base image.

Limitations

Could we push this approach to its utmost potential and generate a talking head from just one or a few images? This task presents a significant challenge. Traditional methods, including deep learning techniques, often introduce noticeable artifacts when applying “warping” transformationsYaroslav Ganin, Daniil Kononenko, Diana Sungatullina, and Victor Lempitsky. 2016. “DeepWarp: Photorealistic Image Resynthesis for Gaze Manipulation.” ArXiv (Cornell University), July. https://doi.org/10.48550/arxiv.1607.07215. to a set of images, thereby diminishing the realism of the outputAverbuch-Elor, Hadar, Daniel Cohen-Or, Johannes Kopf, and Michael F. Cohen. 2017. “Bringing Portraits to Life.” ACM Transactions on Graphics 36 (6): 1–13. https://doi.org/10.1145/3130800.3130818..

Despite recent advances in convolutional ...