Auditory-visual speech perception in normal-hearing and cochlear-implant listeners

Abstract

The present study evaluated auditory-visual speech perception in cochlear-implant users as well as normal-hearing and simulated-implant controls to delineate relative contributions between sensory experience and cues. Auditory-only, visual-only, or auditory-visual speech perception was examined in the context of categorical perception, in which an animated face mouthing /ba/, /da/, or /ga/ was paired with synthesized phonemes from an 11-token auditory continuum. A 3-alternative, forcedchoice, method was used to yield percent identification scores. Normal-hearing listeners showed sharp phoneme boundaries and strong reliance on the auditory cue, whereas actual and simulated implant listeners showed much weaker categorical perception but stronger dependence on the visual cue. The implant users were able to integrate both congruent and incongruent acoustic and optical cues to derive relatively weak but significant auditory-visual integration. This auditory-visual integration was correlated with the duration of the implant experience but not the duration of deafness. Compared with the actual implant performance, acoustic simulations of the cochlear implant could predict the auditory-only performance but not the auditory-visual integration. These results suggest that both altered sensory experience and improvised acoustic cues contribute to the auditory-visual speech perception in cochlear-implant users.

Event-related potentials for better speech perception in noise by cochlear implant users

“…The present results confirmed the previous findings that CI users compensatively use lip-reading to improve speech perception (Kaiser et al, 2003;Rouger et al, 2007Rouger et al, , 2008Desai et al, 2008). The present CI users showed significantly improved word perception performances in all of the quiet and noise conditions in comparisons between the AO and AV conditions.…”

“…The present CI users showed significantly improved word perception performances in all of the quiet and noise conditions in comparisons between the AO and AV conditions. As has been argued in previous studies (Kaiser et al, 2003;Desai et al, 2008), the present CI users used lip-reading despite having more than 2 years of CI use.…”

Audio–visual integration during speech perception in prelingually deafened Japanese children revealed by the McGurk effect

“…The McGurk effect is more induced as age increases [9,20]. The patients using CI showed stronger reliance on visual cues than those of normal-hearing controls by the McGurk stimuli [21][22][23][24].…”

“…Previous studies with the McGurk stimuli have shown similarities in the perception of incongruent audiovisual information in patients with CI and individuals with normalhearing [21][22][23][24]. Desai et al [21] evaluated postlingually deafened English-speaking adult CI users and normal-hearing controls with the McGurk stimuli, and found that CI users showed strong reliance on visual cues whereas normal-hearing controls showed reliance on auditory cues. Rouger et al [22] investigated the McGurk effect in adult French patients and reported that those with CI continued to maintain strong lip reading abilities.…”

“…English, Italian, Spanish) [17][18][19], is easily induced under lower signal-to-noise ratio (S/N ratio) [17], and is more induced as age increase in English-speaking normal-hearing children [9,20]. Previous studies with the McGurk stimuli have shown similarities in the perception of incongruent audiovisual information in patients with CI and individuals with normalhearing [21][22][23][24]. Desai et al [21] evaluated postlingually deafened English-speaking adult CI users and normal-hearing controls with the McGurk stimuli, and found that CI users showed strong reliance on visual cues whereas normal-hearing controls showed reliance on auditory cues.…”