Measuring facial expressions by computer image analysis

Abstract

Facial expressions provide an important behavioral measure for the study of emotion, cognitive processes, and social interaction. The Facial Action Coding System~Ekman & Friesen, 1978! is an objective method for quantifying facial movement in terms of component actions. We applied computer image analysis to the problem of automatically detecting facial actions in sequences of images. Three approaches were compared: holistic spatial analysis, explicit measurement of features such as wrinkles, and estimation of motion flow fields. The three methods were combined in a hybrid system that classified six upper facial actions with 91% accuracy. The hybrid system outperformed human nonexperts on this task and performed as well as highly trained experts. An automated system would make facial expression measurement more widely accessible as a research tool in behavioral science and investigations of the neural substrates of emotion.

The effect of alcohol dependence on automatic visuo-spatial perspective taking

“…Figure 1 provides an example of the facial stimuli that were used from the Karolinska database (Lundqvist et al, 1998).…”

Computer Expression Recognition Toolbox

“…The training set consisted of over 8000 images from both posed and spontaneous expressions, which were coded for facial actions from the Facial Action Coding System. The datasets were the Cohn-Kanade DFAT-504 dataset (Kanade, Cohn & Tian, 2000); The Ekman, Hager dataset of directed facial actions (Bartlett et al, 1999); A subset of 50 videos from 20 subjects from the MMI database (Pantic et al, 2005); and three spontaneous expression datasets collected by Mark Frank D005, D006, D007 (Bartlett et. al.…”

A psycho-ethological approach to social signal processing

“…Whenever possible, the extraction of features should reflect temporal changes in intensity of the measured parameter and should operate on a comparable time window across channels and modalities in order to allow multimodal analysis of social signals. The acquisition of these measurements depends on the tools that are currently being developed towards automatic analysis of facial behaviour (Bartlett et al 1999;Cohn et al 1999; for a review see Pantic and Rothkrantz 2000) and the extraction of vocal parameters (Boersma 2001). Automatic analysis of postures and gestures still represents a challenge for SSP but progresses have been made on the detection of head movements (Kapoor and Picard 2001;Kawato and Ohya 2000;Tan and Rong 2003) as well as hand gestures (Erol et al 2007;Morency et al 2005) and body movements (Oikonomopoulos et al 2009).…”