Interpretation of prosodically marked focus in cochlear implant-simulated speech by non-native listeners

Abstract

This study assesses how a cochlear implant (CI) simulation influences the interpretation of prosodically marked linguistic focus in a non-native language. In an online experiment, two groups of normal-hearing native Dutch learners of English of different ages (12-14 year-old adolescents vs. 18+ year-old adults) and with different proficiency levels in English (A2 vs. B2/C1) were asked to listen to CI-simulated and non-CIsimulated English sentences differing in prosodically marked focus and indicate which of four possible context questions the speaker answered. Results show that, as expected, focus interpretation is significantly less accurate in the CI-simulated condition compared to the non-CI-simulated condition and that more proficient non-native listeners outperform less proficient non-native listeners. However, there was no interaction between the influence of the spectro-temporal degradation of the CIsimulated speech signal and that of the English proficiency level of the non-native listeners, suggesting that less proficient nonnative listeners are not more strongly affected by the spectrotemporal degradation of an electric speech signal than more proficient non-native listeners.

Weighting of Prosodic and Lexical-Semantic Cues for Emotion Identification in Spectrally Degraded Speech and With Cochlear Implants

“…Therefore, the loss of voice pitch information results in significant deficits in CI users' perception of vocal emotions (Luo et al 2007;Hopyan-Misakyan et al 2009;Chatterjee et al 2015;Tinnemore et al 2018;Barrett et al 2020). Consistent with findings in CI listeners, identification of emotional prosody is also impaired in normally hearing listeners subjected to CI-simulated, or vocoded, speech (Shannon et al 1995;Chatterjee et al 2015;Tinnemore et al 2018;Ritter & Vongpaisal 2018;Everhardt et al 2020). Despite CI users' limitations in pitch perception and in identification of prosodic cues, the average CI user shows excellent sentence recognition with high-context materials in quiet environments (James et al 2019).…”

Cortical haemodynamic responses predict individual ability to recognise vocal emotions with uninformative pitch cues but do not distinguish different emotions

“…Previous investigations of cue-weighting in vocal emotions either attenuated variations in individual acoustic features to observe how reducing the information provided by cues impairs accuracy (Gilbers et al, 2015;Luo et al, 2007) or attenuated variations in pairs of acoustic features to quantify listeners' abilities to make use of individual acoustic features providing potentially informative cues. Specifically, they provide preliminary evidence that when F0 information is missing, listeners' may be able to use intensity and/or speech rate cues to glean emotional meaning (e.g., Everhardt et al, 2020;Hegarty & Faulkner, 2013;Metcalfe, 2017), and that intensity cues may be more reliable than speech-rate cues in doing so (e.g., Marx et al, 2015;Peng et al, 2012). Based on these reports, we hypothesise that the accuracy with which vocal emotions are identified is reduced as potential cues, ordered from least to most impactful, are rendered uninformative, that is, intensity and speech-rate cues combined, then F0 cues alone, followed by F0 and intensity cues combined and, finally, F0 and speech-rate cues combined.…”

“…Consistent with the degradation of the speech signal being less severe in hearing with HAs compared to with CIs, HA users tend to show a milder reduction in accuracy when recognising vocal emotion (Most & Aviner, 2009). While increasing evidence demonstrates that HA and CI listeners have difficulty recognising vocal emotions due to degraded F0 information (e.g., Everhardt et al, 2020;Goy et al, 2018;Most & Aviner, 2009;Waaramaa, Kukkonen, Mykkänen, & Geneid, 2018;Waaramaa, Kukkonen, Stoltz, & Geneid, 2018), little is known about neural mechanisms underpinning poor and successful recognition of vocal emotions in hearing-impaired individuals.…”

“…While CIs transmit F0 information weakly, still, compared to other voice-related cues, CI users seem to make more use of F0 cues, albeit to differing degrees for different tasks (Chatterjee & Peng, 2008;Everhardt et al, 2020;Fuller et al, 2014;Gaudrain & Bas ¸kent, 2018;Plack, 2018). F0 is considered to be transmitted through HAs due to the amplification of acoustic signal.…”

“…With F0 playing a primary role as a stand-alone acoustic feature conveying cues to vocal emotions, normal-hearing listeners recognise vocal emotions with variations preserved only in F0, but less so when only intensity or speech-rate cues are preserved (Metcalfe, 2017). Hearing-impaired listeners, including those who use listening devices such as hearing aids (HAs) or cochlear implants (CIs), show difficulties identifying emotions in speech (Chatterjee et al, 2015;Everhardt et al, 2020;Goy et al, 2018;Most & Aviner, 2009), likely the result of reduced spectral resolution (wider auditory filters or channels interactions in implants)-especially for listeners with severe-to-profound hearing loss-and a compressed intensity range in the impaired inner ear (loudness recruitment). No current hearing device is able to entirely compensate for these impairments due to the underlying physiological limitations of hearing loss.…”