Peripheral sounds rapidly activate visual cortex: evidence from electrocorticography

Brang, David; Towle, Vernon L.; Suzuki, Satoru; Hillyard, Steven A.; Tusa, Senneca Di; Dai, Zhongtian; Tao, James X.; Wu, Shasha; Grabowecky, Marcia

doi:10.1152/jn.00728.2015

Cited by 36 publications

(37 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In both cases, the second modality can influence the task-relevant modality as signal transduction times are similar or even faster for the second modality. In our paradigm, auditory stimuli coincided with visual stimuli, which would lead to auditory information reaching the cortex before visual information (Brang et al, 2015). Hence, a phase reset of auditory cortex might not necessarily help ongoing sensory processing.…”

Section: Discussionmentioning

confidence: 98%

“…On the other hand, using MEG, Raij et al . () reported effects in visual cortex due to auditory stimulation already after 50 ms. Concordantly, electrocorticographical (ECoG) results in humans point at a modulation by auditory stimuli in primary visual cortex already after 28 ms (Brang et al ., ). More recently, some authors have linked this auditory co‐activation of visual cortex to an enhancement of visual sensitivity (van der Burg et al ., ; Feng et al ., ).…”

Section: Introductionmentioning

confidence: 97%

See 1 more Smart Citation

The spatio‐temporal profile of multisensory integration

Starke

Ball

Heinze

et al. 2017

Eur J of Neuroscience

View full text Add to dashboard Cite

Task-irrelevant visual stimuli can enhance auditory perception. However, while there is some neurophysiological evidence for mechanisms that underlie the phenomenon, the neural basis of visually induced effects on auditory perception remains unknown. Combining fMRI and EEG with psychophysical measurements in two independent studies, we identified the neural underpinnings and temporal dynamics of visually induced auditory enhancement. Lower- and higher-intensity sounds were paired with a non-informative visual stimulus, while participants performed an auditory detection task. Behaviourally, visual co-stimulation enhanced auditory sensitivity. Using fMRI, enhanced BOLD signals were observed in primary auditory cortex for low-intensity audiovisual stimuli which scaled with subject-specific enhancement in perceptual sensitivity. Concordantly, a modulation of event-related potentials could already be observed over frontal electrodes at an early latency (30-80 ms), which again scaled with subject-specific behavioural benefits. Later modulations starting around 280 ms, that is in the time range of the P3, did not fit this pattern of brain-behaviour correspondence. Hence, the latency of the corresponding fMRI-EEG brain-behaviour modulation points at an early interplay of visual and auditory signals in low-level auditory cortex, potentially mediated by crosstalk at the level of the thalamus. However, fMRI signals in primary auditory cortex, auditory thalamus and the P50 for higher-intensity auditory stimuli were also elevated by visual co-stimulation (in the absence of any behavioural effect) suggesting a general, intensity-independent integration mechanism. We propose that this automatic interaction occurs at the level of the thalamus and might signify a first step of audiovisual interplay necessary for visually induced perceptual enhancement of auditory perception.

show abstract

Section: Discussionmentioning

confidence: 98%

Section: Introductionmentioning

confidence: 97%

The spatio‐temporal profile of multisensory integration

Starke

Ball

Heinze

et al. 2017

Eur J of Neuroscience

View full text Add to dashboard Cite

show abstract

“…The idea that at least the presence of eMSImight be impervious toone's current goals is corroborated by the early (50-100ms) latency of this processand by low-level sensory-perceptual cortices as its likely source (Cappe et al 2010;Raij et al 2000Raij et al , 2010De Meo et al 2015;.Existingresults suggest a surprisingly extensive early cross-talk between inputs from different senses, where auditory-based responses within visual cortices co-occur with or even precede visually-based responses to the same multisensory stimulus (animal models: Schroeder et al 2004;Musacchia and Schroeder 2009;humans: Raij et al 2010;Brang et al 2015). Thus, information is transferred across different senses at latencies still considered as characterising the initial stimulus-driven brain activity, which is thought to be largely independent of top-down control (see e.g.…”

Section: Early Msi As a Hallmark Of A Bottom-up Multisensory Processmentioning

confidence: 98%

The COGs (context, object, and goals) in multisensory processing

et al. 2016

View full text Add to dashboard Cite

Our understanding of how perception operates in real-world environments has been substantially advanced by studying both multisensoryprocesses and "top-down" control processes influencing sensory processing via activity from higher-order brain areas, such as attention, memory, and expectations. As the two topicshave been traditionally studied separately, the mechanisms orchestrating real-world multisensory processingremain unclear.Past work has revealed that the observer's goals gate the influence of many multisensory processes on brain and behavioural responses, whereas some other multisensory processes mightoccur independently of these goals. Consequently, other forms of top-down control beyond goaldependence are necessaryto explain the full range of multisensory effects currently reported at the brain and the cognitive level. These forms of control includesensitivity to stimulus context as well as the detection of matches(or lack thereof) between a multisensory stimulus and categorical attributes of naturalistic objects (e.g. tools, animals). In this review we discuss and integrate the existing findings that demonstrate the importance of such goal-, object-and context-based top-down control over multisensory processing. We then put forward a few principles emerging from this literature review with respect to the mechanisms underlying multisensory processing, and discuss their possible broader implications. Attention, multisensory, control, object, audiovisual, brain mapping 3 Research from the past 30 years has demonstrated a whole range of behavioural benefits engendered by integrating information across the senses (multisensory integration, MSI), including faster motor responses and facilitated object recognition in noisy environments (e.g. Stein 2012). In a separate and independent manner, studies employing unisensory stimuli have been critically advancing our understanding of the nature and the importance of top-down mechanisms that control information processing. Thetopdown nature of these mechanisms lies in that they shape perceptual processing of new inputs by activating information stored in higher-order brain areas (e.g. Summerfield and Egner 2009). Key words:Studies of top-down control have traditionally focused on attentional (i.e. goaldependent)mechanisms, which promote the processing of stimulior objects in the environment that areimportant tothe current behavioural goals of the observer (e.g.Desimone and Duncan 1995). These mechanisms enhance the processing of stimuli appearing in task-relevant spatial locations and/or moments in time. These mechanisms likewise facilitate the processingof those stimuli whose attributes (e.g. colour red), feature dimensions (e.g. shape) or identity (e.g. a particular face) match the observer's goals.Simultaneously, it has been increasingly recognised in the literature that information processing issensitive to other types oftop-down processes, principally those gauged by thememory of past stimulation and one's expectations (see Figure 1A for a summar...

show abstract

“…Current evidence regarding the timing of inputs between sensory cortices has demonstrated a surprising degree of interplay, wherein auditory-driven responses within nominally visual cortices coincide with and sometimes even precede visually-driven responses to the same external event (Schroeder et al, 2004;Musacchia and Schroeder, 2009; see also Raij et al, 2010, andBrang et al, 2015, for MEG and ECoG evidence in humans, respectively).…”

mentioning

confidence: 99%

The context-contingent nature of cross-modal activations of the visual cortex

2016

View full text Add to dashboard Cite

Real-world environments are nearly always multisensory in nature. Processing in such situations confers perceptual advantages, but its automaticity remains poorly understood. Automaticity has been invoked to explain the activation of visual cortices by laterally-presented sounds. This has been observed even when the sounds were task-irrelevant and spatially uninformative about subsequenttargets. An auditory-evoked contralateral occipital positivity (ACOP) at ~250ms post-sound onset has been postulated as the event-related potential (ERP) correlate of this cross-modal effect. However, the spatial dimension of the stimuli was nevertheless relevant in all prior studies where the ACOP was observed. By manipulating the implicit predictability of the location of lateralised sounds in a passive auditory paradigm, we tested the automaticity of cross-modal activations of visual cortices.128-channel ERP data from healthy participants were analysed within an electrical neuroimaging framework. The timing, topography, and localisation resembled previous characterisations of the ACOP. However, the cross-modal activations of visual cortices by sounds were critically dependent on whether the sound location was (un)predictable. Our results are the first direct evidence that this particular cross-modal process is not (fully) automatic; instead, it is context-contingent. More generally, the present findings provide novel insights into the importance of context-related factors in controlling information processing across the senses, and call for a revision of current models of automaticity in cognitive sciences.

show abstract

Peripheral sounds rapidly activate visual cortex: evidence from electrocorticography

Abstract: Peripheral sounds rapidly activate visual cortex: evidence from electrocorticography.

Cited by 36 publications

References 24 publications

The spatio‐temporal profile of multisensory integration

The spatio‐temporal profile of multisensory integration

The COGs (context, object, and goals) in multisensory processing

The context-contingent nature of cross-modal activations of the visual cortex

Contact Info

Product

Resources

About