Deepfakes Overview

Manipulating videos and photographs to edit artifacts has been in practice for quite a long time. In movies like Forrest Gump or Fast and Furious 7, the scenes with John F. Kennedy or Paul Walker in their respective movies were fake and edited into the movies as required.

In the movie Forrest Gump, the scene where Gump meets John F. Kennedy was created using complex visual effects and archival footage to ensure high-quality results. Hollywood studios, spy agencies from across the world, and media outlets have been using editing tools such as Photoshop, After Effects, and complex custom visual effects/CGI (computer-generated imagery) pipelines to come up with such compelling results. While the results have been more or less believable in most instances, it takes a huge amount of manual effort and time to edit each and every detail, such as scene lighting, face, eyes, and lip movements, as well as shadows, for every frame of the scene.

Along the same lines, there is a high chance you might have come across a Buzzfeed videoBuzzFeedVideo. 2018. “You Won’t Believe What Obama Says in This Video!” YouTube Video. YouTube. https://www.youtube.com/watch?v=cQ54GDm1eL0&ab_channel=BuzzFeedVideo. where former US president Barack Obama says, “Killmonger was right” (Killmonger is one of Marvel Cinematic Universe’s villains). While obviously fake, the video does seem real in terms of its visual and audio aspects. There are a number of other examples where prominent personalities can be seen making comments they would usually not.

Keeping ethics aside, there is one major difference between Gump meeting John F. Kennedy and Barack Obama talking about Killmonger. As mentioned earlier, the former is the result of painstaking manual work done using complex visual effects/CGI. The latter, on the other hand, is the result of a technology called deepfakes. A portmanteau of the words deep learning and fake, deepfake is a broad term used to describe AI-enabled technology that is used to generate the examples we discussed.

What are deepfakes?

Deepfakes is an all-encompassing term representing content generated using artificial intelligence (in particular, deep learning) that seems realistic and authentic to a human being. The generation of fake content or manipulation of existing content to suit the needs and agenda of the entities involved is not new. In the introduction, we discussed a few movies where CGI and painstaking manual effort helped in generating realistic results. With advancements in deep learning and, more specifically, generative models, it is becoming increasingly difficult to differentiate between what is real and what is fake.

Generative adversarial networks (GANs) have played a very important role in this space by enabling the generation of sharp, high-quality images and videos. Works based on StyleGAN have really pushed the boundaries in terms of the generation of high-quality realistic content. A number of other key architectures (some of which we discussed earlier) have become key building blocks for different deepfake setups.

Applications of deepfakes

Deepfakes entail realistic-looking content that can be categorized into a number of subcategories. They have a number of applications, which can be categorized into creative, productive, unethical, or malicious use cases.