Advocating a Componential Appraisal Model to Guide Emotion Recognition

Abstract

Most models of automatic emotion recognition use a discrete perspective and a black-box approach, i.e., they output an emotion label chosen from a limited pool of candidate terms, on the basis of purely statistical methods. Although these models are successful in emotion classification, a number of practical and theoretical drawbacks limit the range of possible applications. In this paper, the authors suggest the adoption of an appraisal perspective in modeling emotion recognition. The authors propose to use appraisals as an intermediate layer between expressive features (input) and emotion labeling (output). The model would then be made of two parts: first, expressive features would be used to estimate appraisals; second, resulting appraisals would be used to predict an emotion label. While the second part of the model has already been the object of several studies, the first is unexplored. The authors argue that this model should be built on the basis of both theoretical predictions and empirical results about the link between specific appraisals and expressive features. For this purpose, the authors suggest to use the component process model of emotion, which includes detailed predictions of efferent effects of appraisals on facial expression, voice, and body movements. DOI: 10.4018/jse.2012010102International Journal of Synthetic Emotions, 3(1), 18-32, January-June 2012 19 Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Copyright © 2012, IGI Global. Copying or distributing in print or electronic forms without written permission of IGI Global is prohibited. Mehu, Pantic, & Scherer, in press). Current results are promising and we can expect that in the near future these systems will become fully reliable and perform in a satisfactory way. As the detection problem is getting solved, attention should now focus on what is the best model to attribute an emotional meaning 1 . Indeed, emotion recognition systems can be conceived as made of two parts, a detection component and an inference component. The detection component performs the analysis of the facial movements; the inference component outputs the attribution of an emotional meaning to the movements detected by the first component. While for the detection component there is one recognized standard (FACS), for the inference component we have to turn to emotion psychology where multiple theoretical models currently co-exist. Most researchers in affective computing choose a pragmatic approach and avoid theoretical controversies, but every system necessarily implies theoretical assumptions (Calvo & D'Mello, 2010). In the next paragraphs we will first present different theoretical models of emotion and discuss their use for automatic emotion recognition. The goal is not to provide an exhaustive review of the available systems, but rather a brief description of the pros and cons of each choice. We will then introduce a specific componential appraisal model of emot...

Context in Affective Multiparty and Multimodal Interaction

“…Thus, enriching the gamut of polarized emotions, the representation of emotion intensity will be allowed, as well as the similarity of contrast between various emotions categories and the associated words defining context. c) due to the fact that the detected set of context concepts are expected to operate as mid-level representations, we intend to investigate whether these representations could be used to estimate appraisals and shed light into the first mapping scheme of the componential model which still remains unexplored [25]. Further research on this direction might be able to show whether the combination of this argument is feasible and can introduce a new data acquisition protocol suitable for context.…”

“…Although most AC applications seem to require these major approaches, some have argued that componential approaches [25] might be more appropriate for building affectiveaware frameworks. However, identifying the appropriate level of representation for practical AC applications is still an unresolved question.…”

“…Recently, a research attempt has suggested that another set of psychological models, referred to as componential models of emotion, which are based on the appraisal theory, might be more appropriate for developing context-aware frameworks [25], however, how to use the appraisal approach for automatic analysis of affect is an open research problem. In the componential models of emotion, various ways of linking automatic emotion analysis and appraisal models of emotion are proposed.…”

“…This link aims to enable the addition of contextual information into automatic emotion analyzers, and enrich their interpretation capability in terms of a more sensitive and richer representation. Based on their approach the emotion analysis process is divided into two mapping schemes: expressive features to appraisal variables (first layer) and appraisal variables to emotion label (second layer), providing a number of benefits for automatic emotion analysis [25]. Thus, this latter appraisal based model is more appropriate to the context-aware formalization as it decomposes the appraisal process into the two above layers and deals in a more effective way with the multiparty model's challenges such as the dynamic and bidirectional individual-group relationship and the modeling of context in order to capture more suitable multiparty social setting patterns w.r.t.…”

Behavioral and Physiological Responses to Visual Interest and Appraisals: Multimodal Analysis and Automatic Recognition

“…Recognition of emotions using continuous dimensions, such as valence and arousal, has been also explored but these attempts did not include interest (Hatice et al, 2011;Sariyanidi et al, 2015). Mortillaro et al (2012) proposed that emotion recognition should be done through recognizing cognitive appraisals. If we recognize the appraisals as constructing factors of emotions, we can move beyond the current methods which are mainly based on the automatic recognition of prototypical expressions.…”