Mapping 24 emotions conveyed by brief human vocalization.

Abstract

Emotional vocalizations are central to human social life. Recent studies have documented that people recognize at least 13 emotions in brief vocalizations. This capacity emerges early in development, is preserved in some form across cultures, and informs how people respond emotionally to music. What is poorly understood is how emotion recognition from vocalization is structured within what we call a semantic space, the study of which addresses questions critical to the field: How many distinct kinds of emotions can be expressed? Do expressions convey emotion categories or affective appraisals (e.g., valence, arousal)? Is the recognition of emotion expressions discrete or continuous? Guided by a new theoretical approach to emotion taxonomies, we apply large-scale data collection and analysis techniques to judgments of 2032 emotional vocal bursts produced in laboratory settings (Study 1) and 48 found in the real world (Study 2) by U.S. English speakers (n = 1105). We find that vocal bursts convey at least 24 distinct kinds of emotion. Emotion categories (sympathy, awe) more so than affective appraisals (including valence and arousal) organize emotion recognition. In contrast to discrete emotion theories, the emotion categories conveyed by vocal bursts are bridged by smooth gradients with continuously varying meaning. We visualize the complex, high-dimensional space of emotion conveyed by brief human vocalization within an online interactive map: https://s3-us-west-1.amazonaws.com/vocs/map.html.

Superior communication of positive emotions through nonverbal vocalisations compared to speech prosody

“…Both Dutch and Chinese nonverbal vocalisations were highly effective means of communicating 11 different positive emotions. These results are in line with previous research showing that amusement, interest, lust, relief and surprise are well-recognised from nonverbal vocalisations (e.g., Cowen 2019;Cordaro et al, 2016;Laukka et al, 2013). In addition to these emotions, the current investigation showed that nonverbal vocalisations can reliably communicate admiration, determination, excitement, inspiration, schadenfreude, and sensory pleasure.…”

Emotion recognition and confidence ratings predicted by vocal stimulus type and prosodic parameters

“…Humans communicate emotions with the voice through prosody and vocal bursts (Cowen et al, 2019b). Research has long claimed that certain acoustic features, such as pitch, loudness, tempo or quality and their related parameters (e.g., fundamental frequency, jitter, shimmer, harmonics-to-noise ratio) drive the recognition of emotions from prosody and vocal bursts (e.g., Sauter et al, 2010;Scherer and Baenziger, 2004;Banse and Scherer, 1996).…”

“…An all-encompassing term for such vocal qualities of speech is prosody (i.e., tone of voice). Research has shown that prosody may support the correct interpretations of utterances independently of linguistic comprehension (Paulmann, 2016;Thompson and Balkwill, 2009;Kitayama and Ishii, 2002), with studies reporting recognition rates for emotions to be significantly higher than chance (Cowen et al, 2019a(Cowen et al, , 2019bLausen and Schacht, 2018;Cordaro et al, 2016;Paulmann and Uskul, 2014;Juergens et al, 2013;Scherer et al, 2001). In addition, metacognition, the ability to actively monitor and reflect upon one's own performance, has been argued to impact judgements of accuracy in emotion recognition tasks (Begue et al, 2019;Kelly and Metcalfe, 2011;Dunlosky and Metcalfe, 2009).…”

Autonomous Emotion Learning in Speech: A View of Zero-Shot Speech Emotion Recognition

“…For the label-learning phase, we assume that labellers are able to estimate multiple sets of nA emotional attributes empirically for each of c unseen emotional states. We take this assumption as the first-hand learning for emotions that usually comes from observing physiological or behavioural data [24], indicating that it is natural for different human beings to vocally express one emotional state in various ways [16,17].…”

“…Still, very few of the existing ZSL methods provide a reasonable framework for zero-shot emotion learning in speech. This is due, in part, to the latent emotional descriptors in paralinguistics [6,15] and complicated forms of expression of emotion [16,17].…”