A Multi-Componential Approach to Emotion Recognition and the Effect of Personality

Abstract

Emotions are an inseparable part of human nature affecting our behavior in response to the outside world. Although most empirical studies have been dominated by two theoretical models including discrete categories of emotion and dichotomous dimensions, results from neuroscience approaches suggest a multi-processes mechanism underpinning emotional experience with a large overlap across different emotions. While these findings are consistent with the influential theories of emotion in psychology that emphasize a role for multiple component processes to generate emotion episodes, few studies have systematically investigated the relationship between discrete emotions and a full componential view. This paper applies a componential framework with a data-driven approach to characterize emotional experiences evoked during movie watching. The results suggest that differences between various emotions can be captured by a few (at least 6) latent dimensions, each defined by features associated with component processes, including appraisal, expression, physiology, motivation, and feeling. In addition, the link between discrete emotions and component model is explored and results show that a componential model with a limited number of descriptors is still able to predict the level of experienced discrete emotion(s) to a satisfactory level. Finally, as appraisals may vary according to individual dispositions and biases, we also study the relationship between personality traits and emotions in our computational framework and show that the role of personality on discrete emotion differences can be better justified using the component model.

A Residual Multi-Scale Convolutional Transformer Network with Chunk-level Log-Mel Spectrograms for Speech Emotion Recognition

“…In order to assess the robustness and efficiency of the proposed architecture from a variety of angles, we tested the model on three datasets and conducted a large number of ablation analyses, by this verifying the influence of parameter variables on the predictions. In comparison with earlier studies, we provided a more potent state-of-the-art endto-end model for SER, whose adaptability will encourage the future development of multi-model speech emotion recognition, i. e., by taking advantage of other modalities, such as video and text [68,69,70], Besides, we will also considered how to use chunk-level segments features to create a self-supervised learning framework [71], such as masking some chunk segments during the feature input process and performing contrastive loss on the model output as shown for wav2vec 2.0 [72,73].…”

EmoStim: A Database of Emotional Film Clips with Discrete and Componential Assessment

“…Film clips provide naturalistic events due to their dynamic nature while being easily implemented in the laboratory [19]. They can induce powerful emotions theorized by both discrete [18,19] and dimensional [12,22] models, including complex emotions such as tenderness or compassion [19] beyond basic categories such as fear or disgust. Their effectiveness for studying componential appraisal models [18,43] is also supported by their impact on physiological bodily states such as heart function [13,43], skin conductance [13,18], and brain activity [12,13,22,23].…”

“…They can induce powerful emotions theorized by both discrete [18,19] and dimensional [12,22] models, including complex emotions such as tenderness or compassion [19] beyond basic categories such as fear or disgust. Their effectiveness for studying componential appraisal models [18,43] is also supported by their impact on physiological bodily states such as heart function [13,43], skin conductance [13,18], and brain activity [12,13,22,23]. Our study embraced a wide range of discrete theory-based emotions that were induced using film clips to assess the corresponding CPM descriptors.…”

“…Different stimuli, such as films [12,13], images [14,15], and sounds [16,17], have been used to induce emotions and are generally pre-assigned to different categories according to one given model. Among these stimuli, films have gained wider attention and a reputation in AC as a useful medium to effectively induce diverse emotions [12,18,19]. Features of films, such as cost-effectiveness [20], the possibility to elicit intense emotions [21], easy laboratory setup [19], and triggering of subjective and objective changes [12,13,22,23], can be listed as some advantages.…”

Testing, explaining, and exploring models of facial expressions of emotions

“…Although additional expressor-related facial features may improve prediction performance, they cannot explain differences across individuals (i.e., the variance above the noise ceiling). Therefore, to explain this substantial amount of variance, we must turn to perceiver-related features, which could also be multiple, including the age, gender, sex, personality, and culture of the perceiver, all of which have been shown to influence the interpretation of facial expressions of emotion (47)(48)(49)(50). Our study showed that the perceiver's culture explains part of this variance and that our culture-aware models removed the initial bias towards WE cultures.…”