The temporal cross-capture of audition and vision

Fendrich, Robert; Corballis, Paul M.

doi:10.3758/bf03194432

Cited by 183 publications

(152 citation statements)

References 18 publications

Supporting

Mentioning

138

Contrasting

Order By: Relevance

“…Numerous studies have shown, for example, that perception of a visual event can be modified by the presence of a concurrent sound (Stein et al, 1996;Sekuler et al, 1997;Fendrich and Corballis, 2001;McDonald et al, 2003McDonald et al, , 2005Recanzone, 2003;Vroomen and de Gelder, 2004). A particularly striking perceptual alteration was recently described by Shams et al (2000, who found that presenting a single brief flash interposed between two pulsed sounds separated by 60 -100 ms typically results in the perception of two distinct flashes.…”

Section: Introductionmentioning

confidence: 98%

Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion

Mishra¹,

Martı́nez²,

Sejnowski³

et al. 2007

J. Neurosci.

233

228

View full text Add to dashboard Cite

When a single flash of light is presented interposed between two brief auditory stimuli separated by 60 -100 ms, subjects typically report perceiving two flashes (Shams et al., 2000, 2002). We investigated the timing and localization of the cortical processes that underlie this illusory flash effect in 34 subjects by means of 64-channel recordings of event-related potentials (ERPs). A difference ERP calculated to isolate neural activity associated with the illusory second flash revealed an early modulation of visual cortex activity at 30 -60 ms after the second sound, which was larger in amplitude in subjects who saw the illusory flash more frequently. These subjects also showed this early modulation in response to other combinations of auditory and visual stimuli, thus pointing to consistent individual differences in the neural connectivity that underlies cross-modal integration. The overall pattern of cortical activity associated with the cross-modally induced illusory flash, however, differed markedly from that evoked by a real second flash. A trial-by-trial analysis showed that shortlatency ERP activity localized to auditory cortex and polymodal cortex of the temporal lobe, concurrent with gamma bursts in visual cortex, were associated with perception of the double-flash illusion. These results provide evidence that perception of the illusory second flash is based on a very rapid dynamic interplay between auditory and visual cortical areas that is triggered by the second sound.

show abstract

Section: Introductionmentioning

confidence: 98%

Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion

Mishra¹,

Martı́nez²,

Sejnowski³

et al. 2007

J. Neurosci.

233

228

View full text Add to dashboard Cite

show abstract

“…Here, temporal aspects of a visual stimulus, such as its onset, interval, or duration, can be shifted by slightly asynchronous auditory stimuli (Alais & Burr, 2004a;Bertelson, 1999;Burr, Banks & Morrone, 2009;Fendrich & Corballis, 2001;Freeman & Driver, 2008;Getzmann, 2007;MoreinZamir, Soto-Faraco & Kingstone, 2003;Recanzone, 2009;Scheier, Nijhawan & Shimojo, 1999;Sekuler, Sekuler & Lau, 1997;Vroomen & de Gelder, 2004b; Stetson et al (2006) Shimojo, 2001). One particularly clear manifestation of temporal ventriloquism is that an abrupt sound attracts the apparent onset of a slightly asynchronous flash (see Fig.…”

Section: Temporal Ventriloquism: Immediate Effectmentioning

confidence: 99%

Intersensory binding across space and time: A tutorial review

Chen

Vroomen

2013

Atten Percept Psychophys

211

180

View full text Add to dashboard Cite

Spatial ventriloquism refers to the phenomenon that a visual stimulus such as a flash can attract the perceived location of a spatially discordant but temporally synchronous sound. An analogous example of mutual attraction between audition and vision has been found in the temporal domain, where temporal aspects of a visual event, such as its onset, frequency, or duration, can be biased by a slightly asynchronous sound. In this review, we examine various manifestations of spatial and temporal attraction between the senses (both direct effects and aftereffects), and we discuss important constraints on the occurrence of these effects. Factors that potentially modulate ventriloquism-such as attention, synesthetic correspondence, and other cognitive factors-are described. We trace theories and models of spatial and temporal ventriloquism, from the traditional unity assumption and modality appropriateness hypothesis to more recent Bayesian and neural network approaches. Finally, we summarize recent evidence probing the underlying neural mechanisms of spatial and temporal ventriloquism.

show abstract

“…It is easier to recall an auditory rhythm than it is to recall a visual rhythm (Glenberg et al 1989) and audition affects motor rhythm more than vision (Repp & Penel 2004). The temporal aspects of audio-visual perception are generally considered to be dominated by audition with effects such as temporal capture ( Fendrich & Corballis 2001). Learning of temporal rate discrimination in somatosensation transfers to audition ( Nagarajan et al 1998), suggesting that the extraction of temporal rate is available to other sensory modalities, although plasticity associated with temporal rate learning is observed early in the sensory processing stream (van Wassenhove & Nagarajan 2007).…”

Section: An Amodal Representational Space For Time Perception?mentioning

confidence: 99%

Minding time in an amodal representational space

Wassenhove

2009

Phil. Trans. R. Soc. B

156

111

View full text Add to dashboard Cite

How long did it take you to read this sentence? Chances are your response is a ball park estimate and its value depends on how fast you have scanned the text, how prepared you have been for this question, perhaps your mood or how much attention you have paid to these words. Time perception is here addressed in three sections. The first section summarizes theoretical difficulties in time perception research, specifically those pertaining to the representation of time and temporal processing. The second section reviews non-exhaustively temporal effects in multisensory perception. Sensory modalities interact in temporal judgement tasks, suggesting that (i) at some level of sensory analysis, the temporal properties across senses can be integrated in building a time percept and (ii) the representational format across senses is compatible for establishing such a percept. In the last section, a two-step analysis of temporal properties is sketched out. In the first step, it is proposed that temporal properties are automatically encoded at early stages of sensory analysis, thus providing the raw material for the building of a time percept; in the second step, time representations become available to perception through attentional gating of the raw temporal representations and via re-encoding into abstract representations.

show abstract

The temporal cross-capture of audition and vision

Cited by 183 publications

References 18 publications

Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion

Early Cross-Modal Interactions in Auditory and Visual Cortex Underlie a Sound-Induced Visual Illusion

Intersensory binding across space and time: A tutorial review

Minding time in an amodal representational space

Contact Info

Product

Resources

About