Dynamic Facial Expression of Emotion and Observer Inference

Abstract

Research on facial emotion expression has mostly focused on emotion recognition, assuming that a small number of discrete emotions is elicited and expressed via prototypical facial muscle configurations as captured in still photographs. These are expected to be recognized by observers, presumably via template matching. In contrast, appraisal theories of emotion propose a more dynamic approach, suggesting that specific elements of facial expressions are directly produced by the result of certain appraisals and predicting the facial patterns to be expected for certain appraisal configurations. This approach has recently been extended to emotion perception, claiming that observers first infer individual appraisals and only then make categorical emotion judgments based on the estimated appraisal patterns, using inference rules. Here, we report two related studies to empirically investigate the facial action unit configurations that are used by actors to convey specific emotions in short affect bursts and to examine to what extent observers can infer a person's emotions from the predicted facial expression configurations. The results show that (1) professional actors use many of the predicted facial action unit patterns to enact systematically specified appraisal outcomes in a realistic scenario setting, and (2) naïve observers infer the respective emotions based on highly similar facial movement configurations with a degree of accuracy comparable to earlier research findings. Based on estimates of underlying appraisal criteria for the different emotions we conclude that the patterns of facial action units identified in this research correspond largely to prior predictions and encourage further research on appraisal-driven expression and inference.

Appraisal Theories for Emotion Classification in Text

“…However, some research exists which studies these theories with added value for computational models. Scherer et al (2019), e.g., explicitly encode realizations of appraisals of events and compare those between production and perception of facial expressions. Marsella and Gratch (2009) model emotions in video sequences computationally, based on appraisals of events.…”

Understanding the Multidimensional and Dynamic Nature of Facial Expressions Based on Indicators for Appraisal Components as Basis for Measuring Drivers' Fear

“…The Component Process Model (CPM) provides a comprehensive theoretical framework for the multidimensional and dynamic interpretation of emotions (Scherer, 1984 , 2009a ; Scherer et al, 2019 ). According to the CPM, a given situation would be appraised with multidimensional criteria (the appraisal components), which would follow a fixed order.…”

“…There are four main appraisal components, which were supposed to happen in sequence: 1, novelty, which means, how sudden or unfamiliar the individual perception of the given situation is; 2, pleasantness, which represents positive or negative feelings about the given situation; 3, goal significance, which represents the impact of the situation on an individual goal; 4, coping potential/power, which represents whether the situation is controllable. Particularly, the appraisal components of pleasantness and power are assumed to be the determinants of valence and power suggested by dimensional emotion theorists (Scherer et al, 2019 ). The appraisal component of novelty, on the other hand, is suggested to be an additional dimension to the dimensional emotion space (Scherer, 2009b ).…”

“…The appraisal component of novelty, on the other hand, is suggested to be an additional dimension to the dimensional emotion space (Scherer, 2009b ). In the assumption of the CPM, the results of these appraisal components would specifically impact the autonomic nervous system (e.g., changing in heart rate) and somatic nervous system (e.g., changing in facial expressions or voice) (Scherer et al, 2019 ). Thus, multidimension and dynamics in facial expressions and autonomic nervous system activity can be used as indicators for the presence of certain appraisal processes rendering multidimensional and dynamic interpretation of emotion possible.…”

Negative Mood States Are Related to the Characteristics of Facial Expression Drawing: A Cross-Sectional Study

“…Several previous studies have examined the relationships between facial expressions and mood or emotion in humans ( Darwin, 1872 ; Ekman, 1970 ; Matsumoto and Willingham, 2009 ; Wegrzyn et al, 2017 ; Magdin et al, 2019 ; Sato et al, 2019 ), and mood or emotion changes, such as happiness and anger, modulate facial expressions ( Leppanen et al, 2017 ; Aguado et al, 2019 ; Scherer et al, 2019 ). In particular, some facial expressions are universal, reliable markers of discrete emotions when emotions are aroused, and there is no reason to modify or manage the expression ( Darwin, 1872 ; Matsumoto and Willingham, 2009 ).…”