Effects of Dynamic Aspects of Facial Expressions: A Review

Abstract

Temporal dynamics have been increasingly recognized as an important component of facial expressions. With the need for appropriate stimuli in research and application, a range of databases of dynamic facial stimuli has been developed. The present article reviews the existing corpora and describes the key dimensions and properties of the available sets. This includes a discussion of conceptual features in terms of thematic issues in dataset construction as well as practical features which are of applied interest to stimulus usage. To identify the most influential sets, we further examine their citation rates and usage frequencies in existing studies.General limitations and implications for emotion research are noted and future directions for stimulus generation are outlined.KEYWORDS: facial expression, emotion, dynamic, dataset 3 Revised 9/6/2016 A Review of Dynamic Datasets for Facial Expression ResearchExisting research points towards the benefits of facial motion in emotion perception and recognition. By providing unique information about the direction, quality and speed of motion, dynamic stimuli enhance coherence in the identification of affect, lead to stronger emotion judgments, and facilitate the differentiation between posed and spontaneous expressions (for a review see Krumhuber, Kappas, & Manstead, 2013). In the last two decades, this advantage -paired with the stimuli's greater realism and ecological validity -has led to increased questioning and criticism regarding the use of static images (e.g., Tcherkassof, Bollon, Dubois, Pansu, & Adam, 2007; Wehrle, Kaiser, Schmidt, & Scherer, 2000), with a gradual shift in interest towards dynamic expressions.The trend is reflected in the literature with exponential increases of relevant entries over the past thirty-five years. For example, a Google Scholar search for the word "dynamic face" and related phrases i returned a mere 13 articles in 1980-1989 and 87 articles in 1990-1999. format of recordings, (e) visual or audio-visual modality of stimuli, (f) real human encoders, and (g) individual portrayals (as opposed to emotive interactions; note that some might contain both types).In an attempt to provide useful guidance for the readers of this paper, we classified databases in terms of three fundamental issues that are relevant to decisions about stimulus sets. These include a) conceptual features, which reflect thematic approaches in database construction and validation (Table 1), b) practical features, which concern applied aspects related to stimulus usage (Table 2), and c) citation and usage frequencies of dynamic datasets in the literature (Table 3 ii ), thereby elucidating their respective impact in the field. This latter issue can be categorized according to whether a dataset was used as stimulus material in research with human participants (social sciences) or for the training and testing of machine learning algorithms (computer sciences). With the tables designed to give specific information about each dataset, the accompanying text will focus...

Categorizing identity from facial motion

“…For example, verbal communication and emotional expressions occur via spatially distorting specific facial muscles. It is this continuous series of facial movement that provides an abundance of information necessary for all aspects of social cognition (Krumhuber, Kappas & Manstead, 2013;Stoesz & Jakobson, 2013;Knappmeyer, Thornton & Bülthoff, 2003). Knight and Johnston (1997) were among the first to consider how movement influences the identity recognition of contrasted--reversed famous faces.…”

Emotional facial activation induced by unconsciously perceived dynamic facial expressions

“…Since dynamic aspects of naturallyoccurring facial expressions can play an important role in their interpretation (cf. Krumhuber, Kappas, & Manstead, 2013), the demonstrated effectiveness of this paradigm presents the opportunity to explore the unconscious influences of expressions in a more ecologically valid manner that can include natural transitions between expressions. This could open new possibilities for research into the eliciting factors and consequences of spontaneous facial reactions.…”

“…This raises the question as to whether the influences of dynamic aspects of expressions are dependent on their conscious appreciation or can also result from nonconscious processing. A paradigm permitting effective nonconscious processing of dynamic expressions would help to increase the ecological validity of future experimental findings (Krumhuber, Kappas, & Manstead, 2013).…”

Person perception from changing emotional expressions: primacy, recency, or averaging effect?

“…For example, the effect of the start emotion sadness in Hareli et al's study was examined by comparing a static expression of anger with a dynamic change from sadness to anger. This makes it impossible to infer whether any difference between these conditions is due to the start emotion or the difference between static and dynamic expressions, considering that emotional expressions derive part of their informational value from the fact that they change (Frijda, 1986;Krumhuber et al, 2013;Kuppens, Oravecz, & Tuerlinckx, 2010;Scherer, 2009;Van Kleef, 2016).…”

“…Little is known about how person perception is affected by emotional expression changes, that is, expressions changing from one emotion to another. This is surprising given that facial expressions in real-life interactions are typically dynamic, changing from one state to another (Krumhuber, Kappas, & Manstead, 2013).…”