Audiovisual Speech Recognition With a Cochlear Implant and Increased Perceptual and Cognitive Demands

Abstract

Speech recognition in complex environments involves focusing on the most relevant speech signal while ignoring distractions. Difficulties can arise due to the incoming signal’s characteristics (e.g., accented pronunciation, background noise, distortion) or the listener’s characteristics (e.g., hearing loss, advancing age, cognitive abilities). Listeners who use cochlear implants (CIs) must overcome these difficulties while listening to an impoverished version of the signals available to listeners with normal hearing (NH). In the real world, listeners often attempt tasks concurrent with, but unrelated to, speech recognition. This study sought to reveal the effects of visual distraction and performing a simultaneous visual task on audiovisual speech recognition. Two groups, those with CIs and those with NH listening to vocoded speech, were presented videos of unaccented and accented talkers with and without visual distractions, and with a secondary task. It was hypothesized that, compared with those with NH, listeners with CIs would be less influenced by visual distraction or a secondary visual task because their prolonged reliance on visual cues to aid auditory perception improves the ability to suppress irrelevant information. Results showed that visual distractions alone did not significantly decrease speech recognition performance for either group, but adding a secondary task did. Speech recognition was significantly poorer for accented compared with unaccented speech, and this difference was greater for CI listeners. These results suggest that speech recognition performance is likely more dependent on incoming signal characteristics than a difference in adaptive strategies for managing distractions between those who listen with and without a CI.

Exploring neurocognitive factors and brain activation in adult cochlear implant recipients associated with speech perception outcomes—A scoping review

“…Processing speed is the time required to complete a mental task ( Kail and Salthouse, 1994 ). To test this, one task is used that also assesses attention: the “WJ-IV letter and number pattern matching task and the pair cancelation task.” Furthermore, the “Wechsler Adult Intelligence Scale III (WAIS-III) symbol search.” the “NIH toolbox pattern comparison processing speed test”, the “WAIS-III coding test” and the “WJ-IV numbers reversed test” are used (see Table 5b for an overview) ( Hillyer et al, 2019 ; Mussoi and Brown, 2019 ; Tinnemore et al, 2020 ). Of all of these tests only the pattern comparison test showed a significant relationship with sentences in quiet ( p = 0.006) ( Tinnemore et al, 2020 ).…”

“…They showed that performance on this task significantly differed between better and poorer performers on a word task in quiet (Cohen's d = 0.58, p = 0.037) (Völter et al, 2021). However, the performance did not significantly predict performance for sentence perception in quiet (Tinnemore et al, 2020).…”

“…Furthermore, the "Wechsler Adult Intelligence Scale III (WAIS-III) symbol search." the "NIH toolbox pattern comparison processing speed test", the "WAIS-III coding test" and the "WJ-IV numbers reversed test" are used (see Table 5b for an overview) (Hillyer et al, 2019;Mussoi and Brown, 2019;Tinnemore et al, 2020). Of all of these tests only the pattern comparison test showed a significant relationship with sentences in quiet (p = 0.006) (Tinnemore et al, 2020).…”

The recognition of time-compressed speech as a function of age in listeners with cochlear implants or normal hearing

“…Perhaps a simulation of 6-channel vocoded speech would have better matched performance between the two listener groups. Alternatively, one-to-one matching of listeners by age and performance could be done ( Bhargava et al, 2016 ; Tinnemore et al, 2020 ). Choosing a simulation that perfectly matches performance, however, is complicated by the differences in experience listening to spectrally degraded speech, since performance can change over exposure time (e.g., Rosen et al, 1999 ; Davis et al, 2005 ; Smalt et al, 2013 ; Waked et al, 2017 ).…”

The Relevance of the Availability of Visual Speech Cues During Adaptation to Noise-Vocoded Speech

“…Moreover, speech perception in normal hearing (vocoded) and cochlear implant listeners differs when the spectral degradation is convolved with additional degradation in the input signal, e.g., when the speech is accented. While the speech recognition performance is better in CI over NH (vocoded) listeners when listening to unaccented speech, this pattern of performance reverses when speech is accented (Tinnemore et al, 2020). Future studies could therefore aim to address some of these issues by combining noise-vocoding with pitch shifting, to establish how Running head: Use of visual cues during adaptation to noise-vocoded speech 31 listeners perceive and adapt to a more direct representation of the percept likely experienced by people with a cochlear implant.…”