Recognizing Induced Emotions of Movie Audiences from Multimodal Information

Abstract

Recognizing emotional reactions of movie audiences to affective movie content is a challenging task in affective computing. Previous research on induced emotion recognition has mainly focused on using audiovisual movie content. Nevertheless, the relationship between the perceptions of the affective movie content (perceived emotions) and the emotions evoked in the audiences (induced emotions) is unexplored. In this work, we studied the relationship between perceived and induced emotions of movie audiences. Moreover, we investigated multimodal modelling approaches to predict movie induced emotions from movie content based features, as well as physiological and behavioral reactions of movie audiences. To carry out analysis of induced and perceived emotions, we first extended an existing database for movie affect analysis by annotating perceived emotions in a crowd-sourced manner. We find that perceived and induced emotions are not always consistent with each other. In addition, we show that perceived emotions, movie dialogues, and aesthetic highlights are discriminative for movie induced emotion recognition besides spectators' physiological and behavioral reactions. Also, our experiments revealed that induced emotion recognition could benefit from including temporal information and performing multimodal fusion. Moreover, our work deeply investigated the gap between affective content analysis and induced emotion recognition by gaining insight into the relationships between aesthetic highlights, induced emotions, and perceived emotions.

Can Prediction of Turn-management Willingness Improve Turn-changing Modeling?

“…A high CCC value indicates high agreement between the values of the predicted scores and ground truth. This means that prediction and ground truth values are similar to each other, and general trend changes for both signals are the same [40]. We compared the predictions of pairs of regression models by means of two-sided Wilcoxon signed rank tests at a 0.05 significance level [51].…”

Modeling of Recommendation System Based on Emotional Information and Collaborative Filtering

“…The analyzed color images were then transformed into color measurement values and visualized as a graph. Emotion words were matched according to the highest corresponding measurement values [ 36 , 37 ].…”

Toward Emotion Recognition From Physiological Signals in the Wild: Approaching the Methodological Issues in Real-Life Data Collection

“…Annotating interfaces such as PAGAN (Melhart et al, 2019 ) or CARMA (Girard, 2014 ) (see section 3.3.2.2 for more details on annotation tools) have been developed to collect affect ratings by participants experiencing medias. However, researcher must be careful when using the movie or audio extracts as an emotion induction method, as it was found that perceived emotions by the observer (i.e., interpretation of the movie content, e.g., emotions of a movie character perceived by the observer) do not always agree with induced emotions (i.e., emotions felt by the observer)—both when using video (Muszynski et al, 2019 ) and audio (Kallinen and Ravaja, 2006 ).…”