Recognition of Voice Emotion in School Aged Children with Cochlear Implants

Abstract

This work is based on a part of the first author' master's thesis. Objectives: The cochlear implant (CI) is a device providing congenitally deafened children with access to speech perception. Children with CIs may have good speech perception skills and language development, but still experience difficulties in the perception of prosody based on pitch information. The purpose of this study is to examine the difference between children with CIs and children with normal hearing (NH) in their voice emotion recognition abilities. Methods: The participants were 10 children with CIs and 10 children with NH. Children with CIs all attended regular elementary schools, achieved a typical developmental level on a formal language test, and demonstrated good speech perception abilities (95% in an open set word test). The accuracy of emotion recognition was evaluated by listening to a total of 24 sentences with four different emotions (happiness, sadness, fear, and anger). Results: There were statistically significant differences between groups and also among emotions. Overall, children with CIs showed lower performance than children with NH in recognition of voice emotion (16.1 vs. 23.7). Children with CIs more accurately identified happiness than they did fear (5.1 vs. 2.5). Half of their total identification errors were between fear and sadness (47.36%). Conclusion: The results indicate that children with CIs have difficulties in voice emotion recognition, even if they have no difficulties in speech perception or language development. In error analysis, difficulties in identifying between fear and sadness could be explained by confusion in same valence-arousal space. It is possible that prosodic elements of voice emotions were not sufficiently conveyed through the cochlear implant system. Such results highlight the need for a rehabilitation program addressing these features.

Parameter-Specific Morphing Reveals Contributions of Timbre to the Perception of Vocal Emotions in Cochlear Implant Users

“…At a general level, our findings are in line with earlier reports that CI users perform lower in vocal emotion recognition than NH individuals (e.g., Luo et al 2007 ; Schorr et al 2009 ; Agrawal et al 2013 ; See et al 2013 ; Volkova et al 2013 ; Wiefferink et al 2013 ; Jiam et al 2017 ; Kim & Yoon 2018 ; Paquette et al 2018 ; Tinnemore et al 2018 ; Waaramaa et al 2018 ). However, at a more specific level, when considering contributions of fundamental frequency, timbre, and timing cues to the perception of vocal emotion in CI users, the current findings represent an intriguing contrast to previous research: Our results suggest a greater contribution of timbre than F0, at variance with former reports—originating from gender perception tasks—that CI users cannot dependably use timbre ( Fuller et al 2014a ).…”

“…Underlining the limitations of CIs in transmitting paralinguistic social-communicative vocal signals, previous research suggested general deficits in CI users when perceiving emotions (e.g., Luo et al 2007 ; Schorr et al 2009 ; Agrawal et al 2013 ; See et al 2013 ; Volkova et al 2013 ; Wiefferink et al 2013 ; Jiam et al 2017 ; Kim & Yoon 2018 ; Paquette et al 2018 ; Tinnemore et al 2018 ; Waaramaa et al 2018 ), age ( Skuk et al 2020 ), or gender (e.g., Fu et al 2004 , 2005 ; Kovacić & Balaban 2009 , 2010 ; Meister et al 2009 , 2016 ; Li & Fu 2011 ; Massida et al 2013 ; Fuller et al 2014a ; Hazrati et al 2015 ; Gaudrain & Baskent 2018 ; Skuk et al 2020 ) in other people’s voices. Actually, not only do CI users perform less well than normal-hearing (NH) individuals in perceiving nonverbal social cues, they also seem to employ different perceptual strategies: for example, while NH individuals rely on both timbre and F0 when discriminating a speaker’s sex ( Skuk & Schweinberger 2014 ), CI users rely more on F0 alone ( Massida et al 2013 ; Fuller et al 2014a ; Skuk et al 2020 ).…”

Parameter-specific Morphing Reveals Contributions of Timbre and F0 Cues to the Perception of Voice Gender and Age in Cochlear Implant Users

“…Most studies investigating voice perception in CI users found general deficitssometimes with substantial individual differences -in the ability to perceive voice gender (e.g., Fu, Chinchilla, & Galvin, 2004;Fu, Chinchilla, Nogaki, & Galvin, 2005;Fuller et al, 2014;Gaudrain & Baskent, 2018;Hazrati, Ali, Hansen, & Tobey, 2015;Kovacic & Balaban, 2009Li & Fu, 2011;Massida et al, 2013;Meister, Fursen, Streicher, Lang-Roth, & Walger, 2016;Meister, Landwehr, Pyschny, Walger, & von Wedel, 2009) or emotion (e.g., Agrawal et al, 2013;Jiam, Caldwell, Deroche, Chatterjee, & Limb, 2017;Kim & Yoon, 2018;Paquette et al, 2018;Schorr et al, 2009;Tinnemore, Zion, Kulkarni, & Chatterjee, 2018;Waaramaa, Kukkonen, Mykkanen, & Geneid, 2018). Few studies reported impairments in other aspects of voice perception, such as speaker familiarity or identity (e.g., Gonzalez & Oliver, 2005;Muhler, Ziese, & Verhey, 2017;Vongphoe & Zeng, 2005).…”