Auditory-visual fusion in speech perception in children with cochlear implants

Schorr, Efrat A.; Fox, Nathan A.; Wassenhove, Virginie van; Knudsen, Eric I.

doi:10.1073/pnas.0508862102

Cited by 205 publications

(198 citation statements)

References 19 publications

Supporting

Mentioning

168

Contrasting

Unclassified

Order By: Relevance

“…The progressive engagement of the right PSTG/SMG following auditory deprivation duration is observed in conjunction with the disengagement of the left PSTG/SMG [Lee et al, 2007b], a cortical region that normally underpins phonological processing [Binder and Price, 2001]. The engagement of the right PSTG/SMG hence presumably follows from reduced neural activity in the left homologue reflecting phonological decline when auditory feedback is lacking [Andersson et al, 2001;Lazard et al, 2010b;Lee et al, 2007b;Schorr et al, 2005]. The decline in phonological skills in post-lingual deaf subjects [Anderson et al, 2002;Lazard et al, 2010b] could also arise from more general alterations of auditory memory (not exclusively phonological memory) due to the sparseness of auditory inputs [Andersson et al, 2001].…”

Section: Introductionmentioning

confidence: 99%

Bilateral reorganization of posterior temporal cortices in post‐lingual deafness and its relation to cochlear implant outcome

Lazard

Lee

Truy

et al. 2012

Human Brain Mapping

View full text Add to dashboard Cite

Post-lingual deafness induces a decline in the ability to process phonological sounds or evoke phonological representations. This decline is paralleled with abnormally high neural activity in the right posterior superior temporal gyrus/supramarginal gyrus (PSTG/SMG). As this neural plasticity negatively relates to cochlear implantation (CI) success, it appears important to understand its determinants. We addressed the neuro-functional mechanisms underlying this maladaptive phenomenon using behavioral and functional magnetic resonance imaging (fMRI) data acquired in 10 normal-hearing subjects and 10 post-lingual deaf candidates for CI. We compared two memory tasks where subjects had to evoke phonological (speech) and environmental sound representations from visually presented items. We observed dissociations in the dynamics of right versus left PSTG/SMG neural responses as a function of duration of deafness. Responses in the left PSTG/SMG to phonological processing and responses in the right PSTG/ SMG to environmental sound imagery both declined. However, abnormally high neural activity was observed in response to phonological visual items in the right PSTG/SMG, i.e., contralateral to the zone where phonological activity decreased. In contrast, no such responses (overactivation) were observed in the left PSTG/SMG in response to environmental sounds. This asymmetry in functional adaptation to deafness suggests that maladaptive reorganization of the right PSTG/SMG region is not due to balanced hemispheric interaction, but to a specific take-over of the right PSTG/SMG region by phonological processing, presumably because speech remains behaviorally more relevant to communication than the processing of environmental sounds. These results demonstrate that cognitive long-term alteration of auditory processing shapes functional cerebral reorganization. Hum Brain Mapp 00:000-000,

show abstract

Section: Introductionmentioning

confidence: 99%

Bilateral reorganization of posterior temporal cortices in post‐lingual deafness and its relation to cochlear implant outcome

Lazard

Lee

Truy

et al. 2012

Human Brain Mapping

View full text Add to dashboard Cite

show abstract

“…In normally hearing (NH) subjects, although speechreading performance is very low, the association during development between the auditory and visual speech information is critical for a normal acquisition of multisensory speech perception (11). Speechreading can become extremely crucial in the case of profound deafness because the acquisition of strong skills in speechreading is one of the sensory substitution strategies developed by deaf patients to access speech recognition.…”

mentioning

confidence: 99%

Evidence that cochlear-implanted deaf patients are better multisensory integrators

Rouger

Lagleyre

Fraysse

et al. 2007

Proc. Natl. Acad. Sci. U.S.A.

252

276

View full text Add to dashboard Cite

The cochlear implant (CI) is a neuroprosthesis that allows profoundly deaf patients to recover speech intelligibility. This recovery goes through long-term adaptative processes to build coherent percepts from the coarse information delivered by the implant. Here we analyzed the longitudinal postimplantation evolution of word recognition in a large sample of CI users in unisensory (visual or auditory) and bisensory (visuoauditory) conditions. We found that, despite considerable recovery of auditory performance during the first year postimplantation, CI patients maintain a much higher level of word recognition in speechreading conditions compared with normally hearing subjects, even several years after implantation. Consequently, we show that CI users present higher visuoauditory performance when compared with normally hearing subjects with similar auditory stimuli. This better performance is not only due to greater speechreading performance, but, most importantly, also due to a greater capacity to integrate visual input with the distorted speech signal. Our results suggest that these behavioral changes in CI users might be mediated by a reorganization of the cortical network involved in speech recognition that favors a more specific involvement of visual areas. Furthermore, they provide crucial indications to guide the rehabilitation of CI patients by using visually oriented therapeutic strategies.cochlear implant ͉ deafness ͉ multisensory integration ͉ speech comprehension D espite the apparent division between sensory modalities from the receptors to high cortical levels, we can simultaneously integrate visual and auditory signals resulting in qualitative percepts distinct from those derived from a single unisensory stimulus (1, 2). Furthermore, in cases of precise temporal or spatial congruency between the bisensory stimuli, multisensory integration is expressed at the behavioral level by perceptual improvements by reducing ambiguity (3, 4) and at the neuronal level by enhancing neuronal activity (5). Multisensory integration is also essential for speech recognition, which is based on the simultaneous integration of visual information derived from lip movements and auditory cues produced by the talker (6). The McGurk effect, in which a mismatch between the visual and auditory speech signals is artificially introduced, reveals that the visual information derived from lip movements can strongly influence our auditory perception (7). Although we might not be aware of the relevance of the visual cues for normal speech recognition, the influence of vision becomes convincingly apparent when the auditory information is embedded in noise. In degraded auditory conditions, the visuoauditory presentation leads to higher performance of recognition, when compared with the auditory alone stimulation (8, 9), in a mechanism that mimics an improvement in the acoustic signal-to-noise ratio (SNR) (10).In normally hearing (NH) subjects, although speechreading performance is very low, the association during development between the...

show abstract

“…Infants with normal hearing demonstrate sensitivity to auditory-visual integration processes within the first few months of life (Burnham & Dodd, 2004;Lewkowicz, 2000;Patterson & Werker, 2003). Children with hearing loss who had received a cochlear implant after 30 months of age did not consistently report fused auditory-visual percepts in a McGurk stimuli task (Schorr, Fox, van Wassenhove, & Knudsen, 2005). This suggests that auditory-visual integration deprivation during a critical period affects the ability to associate these multimodal stimuli, as measured by a behavioral response.…”

Section: Considerations For the Erp Resultsmentioning

confidence: 82%

Seeing facial motion affects auditory processing in noise

Gilbert

Lansing

Garnsey

2012

Atten Percept Psychophys

View full text Add to dashboard Cite

Speech perception, especially in noise, may be maximized if the perceiver observes the naturally occurring visual-plus-auditory cues inherent in the production of spoken language. Evidence is conflicting, however, about which aspects of visual information mediate enhanced speech perception in noise. For this reason, we investigated the relative contributions of audibility and the type of visual cue in three experiments in young adults with normal hearing and vision. Relative to static visual cues, access to the talker's phonetic gestures in speech production, especially in noise, was associated with (a) faster response times and sensitivity for speech understanding in noise, and (b) shorter latencies and reduced amplitudes of auditory N1 eventrelated potentials. Dynamic chewing facial motion also decreased the N1 latency, but only meaningful linguistic motions reduced the N1 amplitude. The hypothesis that auditory-visual facilitation is distinct to properties of natural, dynamic speech gestures was partially supported. Keywords Evoked potentials . Multisensory processing . Speech perceptionEveryone experiences difficulty understanding speech at some point, especially in noisy environments or in the presence of a hearing loss. The natural setting for speech communication is in a face-to-face environment where people can both hear the speech and see the face of the talker. Considerable evidence has shown that performance on speech perception tasks improves when information from both auditory and visual modalities is available

show abstract

Auditory-visual fusion in speech perception in children with cochlear implants

Cited by 205 publications

References 19 publications

Bilateral reorganization of posterior temporal cortices in post‐lingual deafness and its relation to cochlear implant outcome

Bilateral reorganization of posterior temporal cortices in post‐lingual deafness and its relation to cochlear implant outcome

Evidence that cochlear-implanted deaf patients are better multisensory integrators

Seeing facial motion affects auditory processing in noise

Contact Info

Product

Resources

About