Investigation of Relationships between Discrete and Dimensional Emotion Models in Affective Picture Databases Using Unsupervised Machine Learning

Abstract

Digital documents created to evoke emotional responses are intentionally stored in special affective multimedia databases, along with metadata describing their semantics and emotional content. These databases are routinely used in multidisciplinary research on emotion, attention, and related phenomena. Affective dimensions and emotion norms are the most common emotion data models in the field of affective computing, but they are considered separable and not interchangeable. The goal of this study was to determine whether it is possible to statistically infer values of emotionally annotated pictures using the discrete emotion model when the values of the dimensional model are available and vice versa. A positive answer would greatly facilitate stimuli retrieval from affective multimedia databases and the integration of heterogeneous and differently structured affective data sources. In the experiment, we built a statistical model to describe dependencies between discrete and dimensional ratings using the affective picture databases NAPS and NAPS BE with standardized annotations for 1356 and 510 pictures, respectively. Our results show the following: (1) there is a statistically significant correlation between certain pairs of discrete and dimensional emotions in picture stimuli, and (2) robust transformation of picture ratings from the discrete emotion space to well-defined clusters in the dimensional space is possible for some discrete-dimensional emotion pairs. Based on our findings, we conclude that a feasible recommender system for affective dataset retrieval can be developed. The software tool developed for the experiment and the results are freely available for scientific and non-commercial purposes.

Knowledge Graph Dataset for Semantic Enrichment of Picture Description in NAPS Database

“…The latter affective ratings were collected from a sample of N = 266 children aged 8-12 years [10]. One of the main important features of the NAPS set as a whole (i.e., with all its extensions) is that it combines a relatively large number of pictures with normative ratings identified according to both dimensional and categorical (discrete) emotion theories [11] compared with other stimuli sets, and it also contains additional multiword semantic descriptions organized into different topics, which are all significant features contributing to the successful construction of stimuli sequences for the elicitation of emotional reactions [7,12].…”

Hybrid Natural Language Processing Model for Sentiment Analysis during Natural Crisis

“…Discrete and dimensional emotion models are commonly considered distinct and mutually independent. However, previous studies have shown that it is possible to establish a correlation between data points in both models by employing ML techniques to identify patterns in a multidimensional space that integrates the semantics and emotions of stimulation [23,24].…”

“…Empirical evidence has demonstrated the presence of statistically significant correlations between specific pairs of discrete and dimensional emotions [23]. Similarly, unsupervised ML methods have demonstrated the ability to transform ratings from the discrete emotion space into distinct clusters in the dimensional space for specific pairs of discrete-dimensional emotions [24].…”

Development of a Universal Validation Protocol and an Open-Source Database for Multi-Contextual Facial Expression Recognition

“…Moreover, statistical analysis methodologies [42,43] were employed to establish a connection between the two ways of interpreting emotions, such as associating discrete emotion labels with corresponding valence and arousal value pairs. For instance, the emotion label "happiness" may be linked to a specific combination of valence and arousal values.…”