Summary: Deepfakes with GANs
Get a quick recap of the major learning points in this chapter.
Deepfakes are a complicated subject, both ethically and technically. In this chapter, we discussed deepfake technology, in general, to start with. We presented an overview of what deepfakes are all about and briefly touched upon a number of productive as well as malicious use cases. We presented a detailed discussion on different modes of operation of different deepfake setups and how each of these impacts the overall believability of generated content. While deepfakes is an all-encompassing term associated with videos, images, audio, text, and so on, we focused on visual use cases only in this chapter.
Given our scope, we discussed various feature sets leveraged by different works in this space. In particular, we discussed the Facial Action Coding System (FACS), 3D Morphable Models (3DMM), and facial landmarks. We also discussed how we could perform facial landmark detection using libraries such as dlib and MTCNN. We then presented a high-level flow of tasks to be performed for a deepfakes pipeline. In conjunction with this, we discussed a few common architectures that are widely used to develop such systems.
The second part of the chapter leveraged this understanding to present two hands-on exercises to develop deepfake pipelines from scratch. We first worked toward developing an autoencoder-based face swapping architecture. Through this exercise, we worked through a step-by-step approach for preparing the dataset, training the networks, and finally generating the swapped outputs. The second exercise involved using the pix2pix GAN to perform a reenactment using live video as the source and Barack Obama as the target. We discussed challenges and ways of overcoming the issues we faced with each of these implementations.
In the final section, we discussed the ethical issues and challenges associated with deepfake architectures. We also touched on a few popular off-the-shelf projects that allow anyone with a computer or a smartphone to generate fake content.
We covered a lot of ground in this chapter and worked on some very exciting use cases.
It is important that we reiterate how vital it is to be careful when we are using technology as powerful as this. The implications and consequences could be very dangerous for the entities involved, so we should be mindful of how this knowledge is used.
Get hands-on with 1400+ tech skills courses.