Audio-visual crossmodal interactions in environmental perception: an fMRI investigation

Belardinelli, Marta Olivetti; Sestieri, Carlo; Matteo, Rosalia Di; Delogu, Franco; Gratta, Cosimo Del; Ferretti, Andrea; Caulo, Massimo; Tartaro, Armando; Romani, Gian Luca

doi:10.1007/s10339-004-0024-0

Cited by 42 publications

(63 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The spatial resolution of the fMRI recording allowed us to study the flow of AV information processing in well defined anatomical substructures, which cannot be specified with the scalprecorded EEG signals because of the well known inverse problem. Previous fMRI studies have focused on the spatial localization of brain regions involved in AV processing, showing AV-related activity in pSTG (Hein et al, 2007), MTG (Beauchamp et al, 2004), visual (Belardinelli et al, 2004;Meienbrock et al, 2007), and inferior frontal regions (Taylor et al, 2006;Hein et al, 2007). Here we were able to uncover the temporal sequence of informative activity in those AV-related regions.…”

Section: Discussionsupporting

confidence: 56%

“…As in many other studies (Beauchamp et al, 2004;Belardinelli et al, 2004;van Atteveldt et al, 2004van Atteveldt et al, , 2007, we investigated AV processing with static images and sounds. There is evidence that different neural networks are involved if dynamic visual stimuli are matched with auditory dynamics (Campbell et al, 1997;Munhall et al, 2002;Calvert and Campbell, 2003).…”

Section: Discussionmentioning

confidence: 99%

“…Previous neuroimaging studies focused on spatially localizing brain regions involved in audiovisual (AV) processing. Various neocortical regions are affected by both auditory and visual inputs (Ghazanfar and Schroeder, 2006), including inferior frontal cortex (IFC) (Belardinelli et al, 2004;Taylor et al, 2006;Hein et al, 2007), the superior temporal sulcus (STS) (Calvert et al, 2000;Beauchamp et al, 2004;Stevenson et al, 2007), and visual (Belardinelli et al, 2004;Meienbrock et al, 2007) and auditory (van Atteveldt et al, 2004(van Atteveldt et al, , 2007Hein et al, 2007) areas. However, temporal aspects of information flow across these different brain regions after AV stimulation were not addressed.…”

Section: Introductionmentioning

confidence: 99%

“…To determine information flow after AV stimulation, we used latency maps obtained in individual subjects' native space. For each subject, we defined anatomical ROIs (see below) in brain regions known to be involved in AV process- ing, including Heschl's gyrus (van Atteveldt et al, 2004(van Atteveldt et al, , 2007, superior temporal gyrus (STG) (Hein et al, 2007), middle temporal gyrus (MTG) (Beauchamp et al, 2004), IFC (Belardinelli et al, 2004;Taylor et al, 2006;Hein et al, 2007), secondary visual cortex [inferior occipital cortex (IOC)] (Belardinelli et al, 2004), and primary visual cortex (V1) (Meienbrock et al, 2007).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Temporal Characteristics of Audiovisual Information Processing

et al. 2008

View full text Add to dashboard Cite

In complex natural environments, auditory and visual information often have to be processed simultaneously. Previous functional magnetic resonance imaging (fMRI) studies focused on the spatial localization of brain areas involved in audiovisual (AV) information processing, but the temporal characteristics of AV information flow in these regions remained unclear. In this study, we used fMRI and a novel information-theoretic approach to study the flow of AV sensory information. Subjects passively perceived sounds and images of objects presented either alone or simultaneously. Applying the measure of mutual information, we computed for each voxel the latency in which the blood oxygenation level-dependent signal had the highest information content about the preceding stimulus. The results indicate that, after AV stimulation, the earliest informative activity occurs in right Heschl's gyrus, left primary visual cortex, and the posterior portion of the superior temporal gyrus, which is known as a region involved in object-related AV integration. Informative activity in the anterior portion of superior temporal gyrus, middle temporal gyrus, right occipital cortex, and inferior frontal cortex was found at a later latency. Moreover, AV presentation resulted in shorter latencies in multiple cortical areas compared with isolated auditory or visual presentation. The results provide evidence for bottom-up processing from primary sensory areas into higher association areas during AV integration in humans and suggest that AV presentation shortens processing time in early sensory cortices.

show abstract

Section: Discussionsupporting

confidence: 56%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Temporal Characteristics of Audiovisual Information Processing

et al. 2008

View full text Add to dashboard Cite

show abstract

“…Importantly, it weighted its connectivity to auditory and visual regions according to its reliability and decisional relevance. Another study by Belardinelli et al [25] indicated increased activity in the inferior frontal sulcus when subjects were confronted with incongruent audiovisual pairs. Similar effects are also present in a recent MEG study by Paraskevopoulos et al [26], while direct recording primate studies confirm the multisensory characteristics of this region at a single neuron level [27][28][29].…”

Section: The Multisensory Brainmentioning

confidence: 99%

Multisensory integration and neuroplasticity in the human cerebral cortex

Paraskevopoulos

Herholz

2013

Translational Neuroscience

View full text Add to dashboard Cite

There is a strong interaction between multisensory processing and the neuroplasticity of the human brain. On one hand, recent research demonstrates that experience and training in various domains modifies how information from the different senses is integrated; and, on the other hand multisensory training paradigms seem to be particularly effective in driving functional and structural plasticity. Multisensory training affects early sensory processing within separate sensory domains, as well as the functional and structural connectivity between uniand multisensory brain regions. In this review, we discuss the evidence for interactions of multisensory processes and brain plasticity and give an outlook on promising clinical applications and open questions.

show abstract

Multisensory Assessment of Acoustic Comfort Aboard Metros: a Virtual Reality Study

Iachini

Maffei

Ruotolo

et al. 2012

Applied Cognitive Psychology

View full text Add to dashboard Cite

In this study, a multisensory methodology is used to assess acoustic comfort aboard different real-world metros by means of subjective annoyance and cognitive performance measures. Two experimental conditions were compared: unimodal versus bimodal. Immersive virtual reality was used to simulate journeys aboard metro coaches. Participants performed four tasks (Rey Test, Verbal Fluency, Backward Counting and Auditory Words Discrimination) while listening to metro sounds (unimodal condition) or while listening to metro sounds within a virtual metro coach (bimodal condition). At the end of each journey, participants reported their degree of noise annoyance. The results showed that cognitive performances were worse in the bimodal than in the unimodal condition. Moreover, the bimodal condition affected negatively the capacity to discriminate words and to count backward. However, reported noise annoyance was higher in the unimodal than bimodal condition. The theoretical and practical implications of these findings are discussed.

show abstract

Audio-visual crossmodal interactions in environmental perception: an fMRI investigation

Cited by 42 publications

References 32 publications

Temporal Characteristics of Audiovisual Information Processing

Temporal Characteristics of Audiovisual Information Processing

Multisensory integration and neuroplasticity in the human cerebral cortex

Multisensory Assessment of Acoustic Comfort Aboard Metros: a Virtual Reality Study

Contact Info

Product

Resources

About