Parameter-Specific Morphing Reveals Contributions of Timbre to the Perception of Vocal Emotions in Cochlear Implant Users

Abstract

Objectives: Research on cochlear implants (CIs) has focused on speech comprehension, with little research on perception of vocal emotions. We compared emotion perception in CI users and normal-hearing (NH) individuals, using parameter-specific voice morphing.Design: Twenty-five CI users and 25 NH individuals (matched for age and gender) performed fearful-angry discriminations on bisyllabic pseudoword stimuli from morph continua across all acoustic parameters (Full), or across selected parameters (F0, Timbre, or Time information), with other parameters set to a noninformative intermediate level.Results: Unsurprisingly, CI users as a group showed lower performance in vocal emotion perception overall. Importantly, while NH individuals used timbre and fundamental frequency (F0) information to equivalent degrees, CI users were far more efficient in using timbre (compared to F0) information for this task. Thus, under the conditions of this task, CIs were inefficient in conveying emotion based on F0 alone. There was enormous variability between CI users, with low performers responding close to guessing level. Echoing previous research, we found that better vocal emotion perception was associated with better quality of life ratings.Conclusions: Some CI users can utilize timbre cues remarkably well when perceiving vocal emotions.

Crossmodal benefits to vocal emotion perception in cochlear implant users

“…As an influential recent review on communication with a CI puts it, “the role of voice emotion perception and production in communication cannot be overstated”, 18 (p. 37). The present research corroborates substantially lower VER skills in adult CI users as a group, compared to NH individuals, in line with earlier findings 34 , 39 , 40 ; see Jiam et al. 18 for a review.…”

“…Research into VER with a CI is challenging because there is typically enormous performance variability between CI users, with some approximating the level of NH individuals and others responding close to guessing levels 18 , 33 , 34 ; for a review, see Jiam et al. 18 In the present two experiments, we studied vocal and AV emotion recognition in two groups of approximately 25 adult CI users who reflected this variability.…”

Enhancing socio-emotional communication and quality of life in young cochlear implant recipients: Perspectives from parameter-specific morphing and caricaturing

“…Although F0 and timbre both contribute to speaker gender perception in normal-hearing listeners, this study revealed that adult CI users exclusively used F0 cues to perceive speaker gender, and did not make efficient use of timbre in this task ( Skuk et al, 2020 ). This might be thought to reflect inefficient transmission of timbre cues by a CI, but another recent study with analogous methods and participant samples seems to exclude this simple interpretation: In a task to perceive vocal emotions, adult CI users were far more efficient to use timbre, compared to F0 information ( von Eiff et al, 2022 ). In addition, both studies revealed that these qualitative differences are seen despite large interindividual differences between CI users.…”

“…At the same time, non-verbal communication skills 1 (e.g., Frühholz and Schweinberger, 2021 ) have been comparatively disregarded. This seems unfortunate, because seminal work has shown only a relatively weak relationship between perceived QoL and speech recognition in CI users ( Huber, 2005 ), and because consistent positive correlations between QoL and abilities to perceive vocal emotions were reported more recently ( Schorr et al, 2009 ; Luo et al, 2018 ; von Eiff et al, 2022 ). These are sometimes larger than those between QoL and speech comprehension, such that the role of vocal emotions in communication can hardly be overstated ( Jiam et al, 2017 ).…”

Musicality – Tuned to the melody of vocal emotions

“…In early emotional voice perception studies, F0 has been considered the perceptually dominant cue, alongside with amplitude and timing parameters (Banse & Scherer, 1996; Juslin & Laukka, 2003). Recent work, however, suggests that timbre can play a central role in voice processing, and vocal emotion perception in particular (Nussbaum, Schirmer, & Schweinberger, 2022; Nussbaum, von Eiff, et al., 2022; Piazza et al., 2018; von Eiff et al., 2022). In fact, some data imply that both F0 and timbre carry unique information for different emotions (Anikin, 2020; Grichkovtsova et al., 2012).…”