Key Feature Set

The human face and body are key entities in this task of fake content generation. While deep learning architectures usually do not require hand-crafted features, a little nudge goes a long way when complex entities are involved. Particularly when dealing with the human face, apart from detecting the overall face in a given image or video, a deepfake solution also needs to focus on the eyes, mouth, and other features.

In this lesson, we’ll briefly cover a few important features leveraged by different deepfake solutions. These are as follows:

• Facial Action Coding System (FACS)

• 3D Morphable Model (3DMM)

• Facial landmarks

We will also undertake a couple of hands-on exercises to better understand these feature sets.

Facial Action Coding System (FACS)

Developed by Carl-Herman Hjortsjö in 1969 and later adopted and refined by Ekamn et al. in 1978, the Facial Action Coding System, or FACS, is an anatomy-based system for understanding facial movements. It is one of the most extensive and accurate coding systems for analyzing facial muscles to understand expressions and emotions.

The figure below depicts a few specific muscle actions and their associated meanings.