Appraisal-driven facial actions as building blocks for emotion inference.

Abstract

Although research on facial emotion recognition abounds, there has been little attention on the nature of the underlying mechanisms. In this article, using a "reverse engineering" approach, we suggest that emotion inference from facial expression mirrors the expression process. As a strong case can be made for an appraisal theory account of emotional expression, which holds that appraisal results directly determine the nature of facial muscle actions, we claim that observers first detect specific appraisals from different facial muscle actions and then use implicit inference rules to categorize and name specific emotions. We report three experiments in which, guided by theoretical predictions and past empirical evidence, we systematically manipulated specific facial action units individually and in different configurations via synthesized avatar expressions. Large, diverse groups of participants judged the resulting videos for the underlying appraisals and/or the ensuing emotions. The results confirm that participants can infer targeted appraisals and emotions from synthesized facial actions based on appraisal predictions. We also report evidence that the ability to correctly interpret the synthesized stimuli is highly correlated with emotion recognition ability as part of emotional competence. We conclude by highlighting the importance of adopting a theory-based experimental approach in future research, focusing on the dynamic unfolding of facial expressions of emotion. (PsycINFO Database Record

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

“…This is in line with recent work that points toward the important role of dynamics and bodily movements to define the nonverbal expression of positive emotions (Mortillaro et al, 2011;Dael et al, 2012;Dukes et al, 2017;Mortillaro and Dukes, 2018). It is important to note, that in terms of appraisals we found correlations with Action Units that are in line with the most recent empirical evidence (Scherer et al, 2018).…”

“…Our results about the relationship between appraisals and Facial Action Units are generally in line with the most recent empirical findings (Scherer et al, 2018). Our study also showed that appraisals are related to eye gaze (longer saccades for stimuli appraised as novel) and that electrodermal responses or fluctuations are also associated with higher level of interest.…”

“…For images, coping, potential, novelty and complexity were better recognized than interest in all unimodal detection procedure and also in the multimodal approach. This pattern of results suggests to consider appraisals as building blocks for emotion expressions and use them as first-level target of emotion detection algorithms De Melo et al, 2014;Scherer et al, 2018). We did not find the same pattern of results for micro-videos, but this is mostly due to the better performance in detecting interest when participants watched micro-videos.…”

“…Based on the suggestions of Mortillaro et al Scherer et al, 2018) we explored the possibility to automatically detect appraisals. We focused our attention on the appraisals that are most relevant for interest and achieved promising results.…”

“…Further support to this approach comes from the recent work of De Melo et al (2014) who conducted a series of experiments in which demonstrated that appraisals are recognized by viewers and mediate the effects of emotion displays on expectations about others' intentions. In their discussion of the implications of these results, the authors argue that appraisal-based-approaches could be very useful to design human-computer-interaction systems (see also Scherer et al, 2018).…”