Perception of Child-Directed Versus Adult-Directed Emotional Speech in Pediatric Cochlear Implant Users

Abstract

Objectives: Cochlear implants (CIs) are remarkable in allowing individuals with severe to profound hearing loss to perceive speech. Despite these gains in speech understanding, however, CI users often struggle to perceive elements such as vocal emotion and prosody, as CIs are unable to transmit the spectro-temporal detail needed to decode affective cues. This issue becomes particularly important for children with CIs, but little is known about their emotional development. In a previous study, pediatric CI users showed deficits in voice emotion recognition with child-directed stimuli featuring exaggerated prosody. However, the large intersubject variability and differential developmental trajectory known in this population incited us to question the extent to which exaggerated prosody would facilitate performance in this task. Thus, the authors revisited the question with both adult-directed and child-directed stimuli. Design: Vocal emotion recognition was measured using both child-directed (CDS) and adult-directed (ADS) speech conditions. Pediatric CI users, aged 7–19 years old, with no cognitive or visual impairments and who communicated through oral communication with English as the primary language participated in the experiment (n = 27). Stimuli comprised 12 sentences selected from the HINT database. The sentences were spoken by male and female talkers in a CDS or ADS manner, in each of the five target emotions (happy, sad, neutral, scared, and angry). The chosen sentences were semantically emotion-neutral. Percent correct emotion recognition scores were analyzed for each participant in each condition (CDS vs. ADS). Children also completed cognitive tests of nonverbal IQ and receptive vocabulary, while parents completed questionnaires of CI and hearing history. It was predicted that the reduced prosodic variations found in the ADS condition would result in lower vocal emotion recognition scores compared with the CDS condition. Moreover, it was hypothesized that cognitive factors, perceptual sensitivity to complex pitch changes, and elements of each child’s hearing history may serve as predictors of performance on vocal emotion recognition. Results: Consistent with our hypothesis, pediatric CI users scored higher on CDS compared with ADS speech stimuli, suggesting that speaking with an exaggerated prosody—akin to “motherese”—may be a viable way to convey emotional content. Significant talker effects were also observed in that higher scores were found for the female talker for both conditions. Multiple regression analysis showed that nonverbal IQ was a significant predictor of CDS emotion recognition scores while Years using CI was a significant predictor of ADS scores. Confusion matrix analyses revealed a dependence of results on specific emotions; for the CDS condition’s female talker, participants had high sensitivity (d’ scores) to happy and low sensitivity to the neutral sentences while for the ADS condition, low sensitivity was found for the scared sentences. Conclusions: In general, participants had higher vocal emotion recognition to the CDS condition which also had more variability in pitch and intensity and thus more exaggerated prosody, in comparison to the ADS condition. Results suggest that pediatric CI users struggle with vocal emotion perception in general, particularly to adult-directed speech. The authors believe these results have broad implications for understanding how CI users perceive emotions both from an auditory communication standpoint and a socio-developmental perspective.

Voice emotion recognition by Mandarin‐speaking pediatric cochlear implant users in Taiwan

“…As it was expected that the cCI would have difficulty in the task, the stimuli were recorded in a child-directed manner. 17 The sensitivity of F0 changes was also tested for the cNH and cCI. All participants gave informed consent prior to participation.…”

Voice Emotion Recognition by Mandarin-Speaking Children with Cochlear Implants

“…In the study of Barrett et al (2020), the accuracy scores for adult-directed speech (50.0% ± 14.4%) and that for child-directed speech (71.7% ± 20.3%) were significantly different ( p < 0.001) in speech emotion recognition task undertook by English-speaking children with CIs. However, among Mandarin-speaking children with CIs, the difference between performance for adult-directed speech and that for child-directed speech did not seem to be as large as that reported in the English-speaking children with CIs (Barrett et al 2020). This may be attributed to the variability in the materials and talkers in the two studies.…”

“…In natural utterances, the mean fundamental frequency of female talkers is higher on average than male ones, which is also found in some studies (titze 1989;Lei et al 2000). Male talkers' emotional expressions are generally less distinguishable than those of female talkers (Luo et al 2007;Simon-thomas et al 2009;Chatterjee et al 2015;Lausen & Schacht 2018;Barrett et al 2020). In addition, we wanted to minimize the test time and reduce fatigue during testing, so as to avoid these factors adversely affecting the results.…”

“…We found that recognition performance of voice emotion recognition was better for children with higher nonverbal IQ. Barrett et al (2020) reported that NVIQ could predict the performance of emotion recognition with child-directed speech among English-speaking children with CIs. thus, cognitive and linguistic processes may be more involved in voice emotion recognition tasks, especially for Mandarin-speaking children with CIs.…”

Weighting of Prosodic and Lexical-Semantic Cues for Emotion Identification in Spectrally Degraded Speech and With Cochlear Implants

“…Acoustic cues to emotional prosody include voice pitch and its changes, intensity, duration or speaking rate, and vocal timbre, but voice pitch is dominant among these cues (Banse & Scherer 1996; Chatterjee et al 2015). Therefore, the loss of voice pitch information results in significant deficits in CI users’ perception of vocal emotions (Luo et al 2007; Hopyan-Misakyan et al 2009; Chatterjee et al 2015; Tinnemore et al 2018; Barrett et al 2020). Consistent with findings in CI listeners, identification of emotional prosody is also impaired in normally hearing listeners subjected to CI-simulated, or vocoded, speech (Shannon et al 1995; Chatterjee et al 2015; Tinnemore et al 2018; Ritter & Vongpaisal 2018; Everhardt et al 2020).…”