Perceptual Cues in Nonverbal Vocal Expressions of Emotion

Abstract

Work on facial expressions of emotions (Calder et al, 2001) and emotionally inflected speech (Banse & Scherer, 1996) has successfully delineated some of the physical properties that underlie emotion recognition. To identify the acoustic cues used in the perception of non-verbal emotional expressions like laugher and screams, an investigation was conducted into vocal expressions of emotion, using non-verbal vocal analogues of the 'basic' emotions (anger, fear, disgust, sadness and surprise; Ekman & Friesen, 1971;Scott et al, 1997), and of positive affective states (Ekman, 1992(Ekman, , 2003Sauter & Scott, 2007). First, the emotional stimuli were categorized and rated to establish that listeners could identify and rate the sounds reliably, and to provide confusion matrices. A principal components analysis of the rating data yielded two underlying dimensions, correlating with the perceived valence and arousal of the sounds. Second, acoustic properties of the amplitude, pitch and spectral profile of the stimuli were measured. A discriminant analysis procedure established that these acoustic measures provided sufficient discrimination between expressions of emotional categories to permit accurate statistical classification. Multiple linear regressions with participants' subjective ratings of the acoustic stimuli showed that all classes of emotional ratings could be predicted by some combination of acoustic measures, and that most emotion ratings were predicted by different constellations of acoustic features. The results demonstrate that, similarly to affective signals in facial expressions and emotionally inflected speech, the perceived emotional character of affective vocalizations can be predicted on the basis of their physical features.

Processing of emotional vocalizations in bilateral inferior frontal cortex

“…According to nonhuman primate studies, the IFC is rather sensitive to higherorder acoustic information (complex object representations) instead of first-order acoustic properties (simple spectro-temporal features) in vocalizations (Cohen et al, 2007). However, discrimination and categorization of vocal expressions depends on specific features (Banse and Scherer, 1996;Lakshminarayanan et al, 2003;Sauter et al, 2010).…”

“…However, it seems that the medium of vocal expressions as a bottom-up factor can also influence this peak distribution. Concerning this medium or nature of vocalizations, vocal expressions of emotions can be purely nonverbal, such as laughs, cries, or screams (Sauter et al, 2010;Simon-Thomas et al, 2009), but they can also be superimposed on semantically neutral Kotz et al, 2003;Leitman et al, 2010) and semantically emotional language (Beaucousin et al, 2007;Mitchell, 2006;Mitchell et al, 2003;Schirmer et al, 2004), as well as on pseudospeech (Bach et al, 2008;Sander et al, 2005). Given this distinction between nonverbal vocal expressions and emotional prosody, it might be assumed that emotional intonations superimposed on speech strongly engage the left IFC, whereas nonverbal expressions activate the right IFC (Meyer et al, 2005).…”

“…Similarly, studies in human infants have shown that the IFC is sensitive to nonverbal features of human speech, especially for emotional intonations (Cheng et al, 2012;Grossmann et al, 2010), suggesting an early ontogenetic sensitivity to conspecific vocalizations. Sensitivity of the human IFC to nonverbal vocalizations might be the final precursor to emotional intonations superimposed on speech utterances, because both share similarities of several features of acoustically encoded emotional cues (Banse and Scherer, 1996;Patel et al, 2011;Sauter et al, 2010) and often appear in the same context. Both nonhuman and human primate studies seem to indicate that the IFC sensitivity implies lateralization changes during ontogenetic maturation and development, resulting in an increasing leftward lateralization.…”

“…Intonations superimposed on speech (Banse and Scherer, 1996;Juslin and Laukka, 2003) and paraverbal speech (Patel et al, 2011) and in nonverbal expressions of emotions (Sauter et al, 2010) have global vocal features (i.e. mean intensity or pitch; spectral center of gravity; harmonics-to-noise ratio, HNR), but also show slow and fast dynamic modulations of features.…”

“…Although nonverbal expressions are different because they are not superimposed on speech or speech-like utterances and are usually less temporarily extended, they might figure as precursors to emotional intonated speech (Arnold and Zuberbuhler, 2006;Belin et al, 2008a;Hauser, 1997; see also Section 4). Accordingly, nonverbal expressions have many acoustic features in common with emotional prosody (Banse and Scherer, 1996;Patel et al, 2011;Sauter et al, 2010), from which listeners infer the emotional state of the speaker. The left IFC thus might also decode emotional cues from nonspeech vocalizations, indicating that left IFC activity is not exclusive to syntacto-semantic speech processing, but serves more general functions during the processing of human utterances.…”