Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex

Amedi, Amir; Stern, William; Camprodon, Joan A.; Bermpohl, Felix; Merabet, Lotfi B.; Rotman, Stephen; Hemond, Christopher C.; Meijer, Peter; Pascual‐Leone, Álvaro

doi:10.1038/nn1912

Cited by 373 publications

(382 citation statements)

References 13 publications

Supporting

Mentioning

342

Contrasting

Unclassified

Order By: Relevance

“…Thirdly, it will be important to examine inter-individual variations in tonotopic representations and their consequences on cross-modal modulation of visual cortex excitability. A fourth, but by no means exhaustive domain, would be to capitalize upon these and related findings to optimize parameters of sensory substitution devices in visually-impaired individuals (e.g., Amedi et al, 2007). In conclusion, the present study provides evidence in support of there being direct projections from lowlevel auditory cortex to primary visual cortex that can impact the excitability of visual neurons and in turn perception/ behavior.…”

Section: Discussionsupporting

confidence: 52%

Contributions of pitch and bandwidth to sound-induced enhancement of visual cortex excitability in humans

Spierer

Manuel

Bueti

et al. 2013

Cortex

View full text Add to dashboard Cite

Section: Discussionsupporting

confidence: 52%

Contributions of pitch and bandwidth to sound-induced enhancement of visual cortex excitability in humans

Spierer

Manuel

Bueti

et al. 2013

Cortex

View full text Add to dashboard Cite

“…We predict that the same principle also applies to other information that is correlated in the auditory and visual domains, such as recognition of emotion from voice and face (38,39). Furthermore, this scheme might be a general principle of how unisensory tasks are performed when one or more of the usual input modalities are missing (8,40).…”

Section: Discussionmentioning

confidence: 83%

Simulation of talking faces in the human brain improves auditory speech recognition

Kriegstein

Doğan

Grüter

et al. 2008

Proc. Natl. Acad. Sci. U.S.A.

140

199

View full text Add to dashboard Cite

Human face-to-face communication is essentially audiovisual. Typically, people talk to us face-to-face, providing concurrent auditory and visual input. Understanding someone is easier when there is visual input, because visual cues like mouth and tongue movements provide complementary information about speech content. Here, we hypothesized that, even in the absence of visual input, the brain optimizes both auditory-only speech and speaker recognition by harvesting speaker-specific predictions and constraints from distinct visual face-processing areas. To test this hypothesis, we performed behavioral and neuroimaging experiments in two groups: subjects with a face recognition deficit (prosopagnosia) and matched controls. The results show that observing a specific person talking for 2 min improves subsequent auditory-only speech and speaker recognition for this person. In both prosopagnosics and controls, behavioral improvement in auditory-only speech recognition was based on an area typically involved in face-movement processing. Improvement in speaker recognition was only present in controls and was based on an area involved in face-identity processing. These findings challenge current unisensory models of speech processing, because they show that, in auditory-only speech, the brain exploits previously encoded audiovisual correlations to optimize communication. We suggest that this optimization is based on speaker-specific audiovisual internal models, which are used to simulate a talking face.fMRI ͉ multisensory ͉ predictive coding ͉ prosopagnosia H uman face-to-face communication works best when one can watch the speaker's face (1). This becomes obvious when someone speaks to us in a noisy environment, in which the auditory speech signal is degraded. Visual cues place constraints on what our brain expects to perceive in the auditory channel. These visual constraints improve the recognition rate for audiovisual speech, compared with auditory speech alone (2). Similarly, speaker identity recognition by voice can be improved by concurrent visual information (3). Accordingly, audiovisual models of human voice and face perception posit that there are interactions between auditory and visual processing streams (Fig. 1A) (4, 5).Based on prior experimental (6-8) and theoretical work (9-12) we hypothesized that, even in the absence of visual input, the brain optimizes auditory-only speech and speaker recognition by harvesting predictions and constraints from distinct visual face areas (Fig. 1B).Experimental studies (6,8) demonstrated that the identification of a speaker by voice is improved after a brief audiovisual experience with that speaker (in contrast to a matched control condition). The improvement effect was paralleled by an interaction of voice and face-identity sensitive areas (8). This finding suggested that the associative representation of a particular face facilitates the recognition of that person by voice. However, it is unclear whether this effect also extends to other audiovisual dependencies in human ...

show abstract

“…First, in humans, the detection or discrimination of multisensory objects, as well as the perceptual expertise of subjects, differentially affect the temporal aspects at which multisensory interactions occur (Fort et al, 2002). Second, in blind patients, the cross-modal compensation that occurs following sensory deprivation (Bavelier and Neville, 2002;Bavelier et al, 2006) is expressed as a colonization of the visual areas by somatosensory processing during Braille reading but mainly during an active discrimination task (Sadato and Hallett, 1999; see also Amedi et al, 2007).…”

Section: Electrophysiological Evidence In the Primary Sensory Areasmentioning

confidence: 99%

Multisensory anatomical pathways

2009

View full text Add to dashboard Cite

In order to interact with the multisensory world that surrounds us, we must integrate various sources of sensory information (vision, hearing, touch. . .). A fundamental question is thus how the brain integrates the separate elements of an object defined by several sensory components to form a unified percept. The superior colliculus was the main model for studying multisensory integration. At the cortical level, until recently, multisensory integration appeared to be a characteristic attributed to high-level association regions. First, we describe recently observed direct cortico-cortical connections between different sensory cortical areas in the non-human primate and discuss the potential role of these connections. Then, we show that the projections between different sensory and motor cortical areas and the thalamus enabled us to highlight the existence of thalamic nuclei that, by their connections, may represent an alternative pathway for information transfer between different sensory and/or motor cortical areas. The thalamus is in position to allow a faster transfer and even an integration of information across modalities. Finally, we discuss the role of these non-specific connections regarding behavioral evidence in the monkey and recent electrophysiological evidence in the primary cortical sensory areas.

show abstract

Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex

Cited by 373 publications

References 13 publications

Contributions of pitch and bandwidth to sound-induced enhancement of visual cortex excitability in humans

Contributions of pitch and bandwidth to sound-induced enhancement of visual cortex excitability in humans

Simulation of talking faces in the human brain improves auditory speech recognition

Multisensory anatomical pathways

Contact Info

Product

Resources

About