More Than Words: the Relative Roles of Prosody and Semantics in the Perception of Emotions in Spoken Language by Postlingual Cochlear Implant Users

Abstract

Objectives:The processing of emotional speech calls for the perception and integration of semantic and prosodic cues. Although cochlear implants allow for significant auditory improvements, they are limited in the transmission of spectro-temporal fine-structure information that may not support the processing of voice pitch cues. The goal of the current study is to compare the performance of postlingual cochlear implant (CI) users and a matched control group on perception, selective attention, and integration of emotional semantics and prosody.Design: Fifteen CI users and 15 normal hearing (NH) peers (age range, 18-65 years) 1istened to spoken sentences composed of different combinations of four discrete emotions (anger, happiness, sadness, and neutrality) presented in prosodic and semantic channels-T-RES: Test for Rating Emotions in Speech. In three separate tasks, listeners were asked to attend to the sentence as a whole, thus integrating both speech channels (integration), or to focus on one channel only (rating of target emotion) and ignore the other (selective attention). Their task was to rate how much they agreed that the sentence conveyed each of the predefined emotions. In addition, all participants performed standard tests of speech perception. Results: When asked to focus on one channel, semantics or prosody, both groups rated emotions similarly with comparable levels of selective attention. When the task was called for channel integration, group differences were found. CI users appeared to use semantic emotional information more than did their NH peers. CI users assigned higher ratings than did their NH peers to sentences that did not present the target emotion, indicating some degree of confusion. In addition, for CI users, individual differences in speech comprehension over the phone and identification of intonation were significantly related to emotional semantic and prosodic ratings, respectively.Conclusions: CI users and NH controls did not differ in perception of prosodic and semantic emotions and in auditory selective attention. However, when the task called for integration of prosody and semantics, CI users overused the semantic information (as compared with NH). We suggest that as CI users adopt diverse cue weighting strategies with device experience, their weighting of prosody and semantics differs from those used by NH. Finally, CI users may benefit from rehabilitation strategies that strengthen perception of prosodic information to better understand emotional speech.

Lexical effects on talker discrimination in adult cochlear implant users

“…We expected that adult CI users would utilize linguistic information as a compensatory strategy in talker discrimination, given that they must rely on degraded talker-specific cues. Previous studies have demonstrated that CI users make use of linguistic content (e.g., semantic or lexical content) of an utterance to facilitate spoken word or sentence recognition ( Moberly and Reed, 2019 ; O'Neill et al , 2021 ; Taitelbaum-Swead et al , 2022 ; Tamati and Moberly, 2022 ). Although fewer studies have examined the effect of linguistic content on indexical processing, Richter and Chatterjee (2021) found CI users displayed increased reliance on lexical–semantic content and reduced reliance on prosodic cues in emotion identification, compared with NH listeners.…”

Processing emotional prosody in a foreign language: the case of German and Hebrew

“…In the T-RES, participants are asked to rate sentences spoken with emotional prosody four times, once for each of four discrete emotions (i.e., anger, fear, happiness, sadness). This tool has also been used to compare different groups of participants e.g., older vs. younger adults, participants with high-functioning autism spectrum disorders vs. participants with typical development, participants with tinnitus vs. participants with normal hearing; (Ben-David et al, 2019 , 2020 , 2016 ; Dor et al, 2022a , 2022b ; Leshem et al, 2020 , 2022 ; Oron et al, 2020 ; Taitelbaum-Swead et al, 2022 ). In the current study, we employed the T-RES to compare emotional prosody ratings across speakers of different languages.…”