A novel approach to study audiovisual integration in speech perception: Localizer fMRI and sparse sampling

Szycik, Gregor R.; Tausche, Peggy; Münte, Thomas F.

doi:10.1016/j.brainres.2007.08.027

Cited by 38 publications

(46 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The caudal part of STS has been identified as a multisensory integrative site involved in processing auditory as well as visual stimuli [Beauchamp et al, 2004a,b;Callan et al, 2003;Wright et al, 2003]. In particular, this region seems to be involved in the processing of AV speech [Beauchamp, 2005;Bernstein et al, 2008;Szycik et al, 2007]. Calvert et al [2000] found in the vicinity of the STS, a functional activity cluster with stronger responses for congruent AV speech stimuli.…”

Section: Roi-based Approachmentioning

confidence: 92%

“…Depending on the experimental settings and methods used, different cortical areas appear to be involved in such AV speech processing [for reviews see : Calvert, 2001;Calvert and Thesen, 2004]. There is good evidence for participation of heteromodal cortex centered upon the sulcus temporalis superior (STS) [Beauchamp et al, 2004a,b;Reale et al, 2007;Szycik et al, 2007]. The caudal part of STS revealed different activation for AV speech compared with their unimodal components presented separately.…”

Section: Introductionmentioning

confidence: 98%

See 1 more Smart Citation

Audiovisual integration during speech comprehension: An fMRI study comparing ROI‐based and whole brain analyses

Szycik

Jansma

Münte

2008

Human Brain Mapping

Self Cite

View full text Add to dashboard Cite

Visual information (lip movements) significantly contributes to speech comprehension raising the question for the neural implementation of audiovisual (AV) integration during speech processing. To replicate and extend earlier neuroimaging findings, we compared two different analysis approaches in a slow event-related fMRI study of healthy native speakers of German who were exposed to AV speech stimuli (disyllabic nouns) with audio and visual signals being either congruent or incongruent. First, data was subjected to whole brain general linear model analysis after transformation of all individual data sets into standard space. Second, a region of interest (ROI) approach based on individual anatomy was used with ROI defined in areas identified previously as being important for AV processing. Standard space analysis revealed a widespread cortical network including the posterior part of the left superior temporal sulcus, Broca's region and its right hemispheric counterpart showing increased activity for incongruent stimuli. The ROI approach allowed to identify differences in activity between Brodmann areas 44 and 45, within Broca's area for incongruent stimulation, and also allowed to study activity of subdivisions of superior temporal regions. The complementary strengths and weaknesses of the two analysis approaches are discussed.

show abstract

Section: Roi-based Approachmentioning

confidence: 92%

Section: Introductionmentioning

confidence: 98%

Audiovisual integration during speech comprehension: An fMRI study comparing ROI‐based and whole brain analyses

Szycik

Jansma

Münte

2008

Human Brain Mapping

Self Cite

View full text Add to dashboard Cite

show abstract

“…In a follow up study, this group also reported super-additive responses in the temporal, occipital, parietal and frontal lobes, whilst focusing there discussion on left posterior superior temporal sulcus, as it also displayed a congruency effect (Calvert et al, 2000). Using the max criteria, others (Kreifelts et al, 2007;Szycik et al, 2008;Wright et al, 2003) have found bilateral superior temporal cortex (STC) to be loci of face-voice integration.…”

Section: Introductionmentioning

confidence: 92%

“…This adds some support to the proposal of Joassin et al (2011b), that the hippocampus is a key region in the integration of faces and voices. Also using the max criteria, Szycik et al (2008) found bilateral superior temporal sulcus to be involved in face-voice integration. Two possible reasons why we do not find temporal cortex to pass the max criteria while they do, are stimulus related.…”

Section: Bimodal Face-voice Perceptionmentioning

confidence: 99%

“…For example, manipulations of the congruency (e.g. Calvert et al, 2001;Szycik et al, 2008) and signal strength of stimulus cues (Stevenson and James, 2009) have implicated STC as a site of audiovisual speech integration. Analysing the connectivity between regions found involved in the integration of face and voice has also helped to understand the cerebral mechanisms involved (e.g.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Cerebral Correlates and Statistical Criteria of Cross-Modal Face and Voice Integration

Love

Pollick

Latinus

2011

Seeing and Perceiving

View full text Add to dashboard Cite

Perception of faces and voices plays a prominent role in human social interaction, making multisensory integration of cross-modal speech a topic of great interest in cognitive neuroscience. How to define potential sites of multisensory integration using functional magnetic resonance imaging (fMRI) is currently under debate, with three statistical criteria frequently used (e.g. super-additive, max and mean criteria). In the present fMRI study, 20 participants were scanned in a block design under three stimulus conditions: dynamic unimodal face, unimodal voice and bimodal face-voice. Using this single dataset, we examine all these statistical criteria in an attempt to define loci of face-voice integration. While the super-additive and mean criteria essentially revealed regions in which one of the unimodal responses was a deactivation, the max criteria appeared stringent and * To whom correspondence should be addressed. E-mail: s.love@psy.gla.ac.uk 1 only highlighted the left hippocampus as a potential site of face-voice integration. Psychophysiological interaction analysis showed that connectivity between occipital and temporal cortices increased during bimodal compared to unimodal conditions. We concluded that, when investigating multisensory integration with fMRI, all these criteria should be used in conjunction with manipulation of stimulus signal-to-noise ratio and/or cross-modal congruency.

show abstract

An ALE meta‐analysis on the audiovisual integration of speech signals

Erickson

Heeg

Rauschecker

et al. 2014

Human Brain Mapping

View full text Add to dashboard Cite

The brain improves speech processing through the integration of audiovisual (AV) signals. Situations involving AV speech integration may be crudely dichotomized into those where auditory and visual inputs contain 1) equivalent, complementary signals (validating AV speech), or 2) inconsistent, different signals (conflicting AV speech). This simple framework may allow for the systematic examination of broad commonalities and differences between AV neural processes engaged by various experimental paradigms frequently used to study AV speech integration. We conducted an activation likelihood estimation (ALE) meta-analysis of 22 functional imaging studies comprising 33 experiments, 311 subjects, and 347 foci examining “conflicting” versus “validating” AV speech. Experimental paradigms included content congruency, timing synchrony, and perceptual measures, such as the McGurk effect or synchrony judgments, across AV speech stimulus types (sub-lexical to sentence). Co-localization of conflicting AV speech experiments revealed consistency across at least two contrast types (e.g., synchrony and congruency) in a network of dorsal-stream regions in the frontal, parietal, and temporal lobes. There was consistency across all contrast types (synchrony, congruency, and percept) in the bilateral posterior superior/middle temporal cortex. Although fewer studies were available, validating AV speech experiments were localized to other regions, such as ventral-stream visual areas in the occipital and inferior temporal cortex. These results suggest that while equivalent, complementary AV speech signals may evoke activity in regions related to the corroboration of sensory input, conflicting AV speech signals recruit widespread dorsal-stream areas likely involved in the resolution of conflicting sensory signals.

show abstract

A novel approach to study audiovisual integration in speech perception: Localizer fMRI and sparse sampling

Cited by 38 publications

References 38 publications

Audiovisual integration during speech comprehension: An fMRI study comparing ROI‐based and whole brain analyses

Audiovisual integration during speech comprehension: An fMRI study comparing ROI‐based and whole brain analyses

Cerebral Correlates and Statistical Criteria of Cross-Modal Face and Voice Integration

An ALE meta‐analysis on the audiovisual integration of speech signals

Contact Info

Product

Resources

About