Within and Across-Language Comparison of Vocal Emotions in Mandarin and English

Abstract

This study reports experimental results on whether the acoustic realization of vocal emotions differs between Mandarin and English. Prosodic cues, spectral cues and articulatory cues generated by electroglottograph (EGG) of five emotions (anger, fear, happiness, sadness and neutral) were compared within and across Mandarin and English through a production experiment. Results of within-language comparison demonstrated that each vocal emotion had specific acoustic patterns in each language. Moreover, normalized data were used in the across-language comparison analysis. Results indicated that Mandarin and English showed different mechanisms of utilizing pitch for encoding emotions. The differences in pitch variation between neutral and other emotions were significantly larger in English than in Mandarin. However, the variations of speech rate and certain phonation cues (e.g., CPP (Cepstral Peak Prominence) and CQ (Contact quotient)) were significantly greater in Mandarin than in English. The differences in emotional speech between the two languages may be due to the restriction of pitch variation by the presence of lexical tones in Mandarin. This study reveals an interesting finding that occurs when a certain cue (e.g., pitch) is restricted in one language, other cues were strengthened to take on the responsibility of differentiating vocal emotions. Therefore, we posit that the acoustic realizations of emotional speech are multidimensional.

Emotional tones of voice affect the acoustics and perception of Mandarin tones

“…Studies using non-tonal languages showed that sad and angry voices are easier to identify than fearful and happy voices [2,4,7,50,[75][76][77]. Studies using Mandarin have reported similar findings: negative emotions such as sadness [41,46,66] and fear [43] are easier to identify than positive emotions such as happiness [41,43]. It has been suggested that negative emotions are prioritized in vocal communication because they convey warnings in situations of attack, loss, and danger.…”

“…Wang, Lee, and Ma (2016, 2018) [ 46 , 66 ] examined acoustic correlates of angry, fear, happy, sad, and neutral emotions in Mandarin and English. Semantically-neutral declarative sentences were embedded in different contexts to elicit angry, fear, happy, sad, and neutral emotions.…”

Recognition of affective prosody in autism spectrum conditions: A systematic review and meta-analysis

“…Given the existence of distinctive acoustic features of emotional intonation in a tonal language such as Chinese (Li et al, 2011;T. Wang et al, 2014T. Wang et al, , 2018, it remains to be examined whether emotional processing would also demonstrate language-specific effects in individuals with ASC.…”

A Crosslinguistic Study of the Perception of Emotional Intonation

“…Only recently, emotional intonation in the context of communication between L1 users and LX users has been investigated due to its communicative value in interactions. These studies apply different methods, such as acoustic analysis (e.g., Graham & Post, 2018;Verdugo, 2005;Wang et al, 2018;Wennerstorm, 2001), psycholinguistic tasks (e.g., Kitayama & Ishii, 2002;Pell & Skorup, 2008), or perception-judgment tasks (e.g., Graham et al, 2001;Min & Schirmer, 2011) to investigate the different aspects of emotional speech.…”

“…Currently, there is a growing interest in the acoustic characteristics of emotional speech (e.g., Ish & Kanda, 2019;Min & Schirmer, 2011;Wang et al, 2018) and the perception of emotions by L1 and LX users 1 (Dewaele et al, 2019;Lorette & Dewaele, 2015. This is partly because using a language for communicating emotions is not only essential but also unique to human communication.…”