Multi-score Learning for Affect Recognition: The Case of Body Postures

Abstract

Abstract. An important challenge in building automatic affective state recognition systems is establishing the ground truth. When the groundtruth is not available, observers are often used to label training and testing sets. Unfortunately, inter-rater reliability between observers tends to vary from fair to moderate when dealing with naturalistic expressions. Nevertheless, the most common approach used is to label each expression with the most frequent label assigned by the observers to that expression. In this paper, we propose a general pattern recognition framework that takes into account the variability between observers for automatic affect recognition. This leads to what we term a multi-score learning problem in which a single expression is associated with multiple values representing the scores of each available emotion label. We also propose several performance measurements and pattern recognition methods for this framework, and report the experimental results obtained when testing and comparing these methods on two affective posture datasets.

Intelligent affect regression for bodily expressions using hybrid particle swarm optimization and adaptive ensembles

“…For example, Kleinsmith et al (2011) measured agreement of annotators by iteratively comparing each pair of them. Meng et al (2011) applied multi-labeling techniques that attempted to model the ranking of preferences instead of an absolute judgment, and thus can reduce the noise caused by a forced choice annotation approach.…”

Laughter Type Recognition from Whole Body Motion

“…In order to robustly evaluate the multiple outcomes of the models against the distribution of the observers categorisations, as suggested in [23], we apply four well established multiscore metrics over a number of instances M : 1) Mean Square Error: this is the standard loss function which is computed as:…”

“…There has been growing evidence supporting the possibility of automatically discriminating between different emotions from various modalities: acoustics [17], facial expressions [18] and body [19]- [23]. Moreover, the study in [24] went further in trying to characterize different types of laughter.…”

Using a Probabilistic Topic Model to Link Observers' Perception Tendency to Personality

“…Of particular note, some researchers tried to estimate the distribution of perception among observers, i.e. how emotions are differently perceived by observers for specific stimuli (human behavior) [7], [8]. However, none of them addressed the modeling of the perception tendency of a specific observer.…”

“…To alleviate the subjectivity of perceivers, most previous works gathered the perceptions of multiple observers, and targeted their representative value, e.g. the majority/peak [5] or mean [6], or the distribution [7], [8]. To gather objective descriptions of emotions, observers unacquainted with the target people are often employed, like [7].…”