Sound-Induced Enhancement of Low-Intensity Vision: Multisensory Influences on Human Sensory-Specific Cortices and Thalamic Bodies Relate to Perceptual Enhancement of Visual Detection Sensitivity

Noesselt, Toemme; Tyll, Sascha; Boehler, C. Nico; Budinger, Eike; Heinze, Hans‐Jochen; Driver, Jon

doi:10.1523/jneurosci.4524-09.2010

Cited by 143 publications

(166 citation statements)

References 100 publications

(159 reference statements)

Supporting

Mentioning

142

Contrasting

Order By: Relevance

“…5). This further highlights the behavioral relevance of early-latency and low-level multisensory interactions in humans (Romei et al, 2007(Romei et al, , 2009Sperdin et al, 2009Sperdin et al, , 2010Noesselt et al, 2010;Van der Burg et al, 2011) as well as monkeys (Wang et al, 2008). Such a linear relationship also provides further support to the suggestion that looming signals are on the one hand preferentially processed neurophysiologically (Maier et al, 2008) and on the other hand subject to perceptual biases (Maier et al, 2004).…”

Section: Mechanisms Subserving the Integration Of Looming Signalssupporting

confidence: 63%

Looming Signals Reveal Synergistic Principles of Multisensory Integration

Cappe¹,

Thelen²,

Romei³

et al. 2012

J. Neurosci.

View full text Add to dashboard Cite

Multisensory interactions are a fundamental feature of brain organization. Principles governing multisensory processing have been established by varying stimulus location, timing and efficacy independently. Determining whether and how such principles operate when stimuli vary dynamically in their perceived distance (as when looming/receding) provides an assay for synergy among the above principles and also means for linking multisensory interactions between rudimentary stimuli with higher-order signals used for communication and motor planning. Human participants indicated movement of looming or receding versus static stimuli that were visual, auditory, or multisensory combinations while 160-channel EEG was recorded. Multivariate EEG analyses and distributed source estimations were performed. Nonlinear interactions between looming signals were observed at early poststimulus latencies (ϳ75 ms) in analyses of voltage waveforms, global field power, and source estimations. These looming-specific interactions positively correlated with reaction time facilitation, providing direct links between neural and performance metrics of multisensory integration. Statistical analyses of source estimations identified looming-specific interactions within the right claustrum/insula extending inferiorly into the amygdala and also within the bilateral cuneus extending into the inferior and lateral occipital cortices. Multisensory effects common to all conditions, regardless of perceived distance and congruity, followed (ϳ115 ms) and manifested as faster transition between temporally stable brain networks (vs summed responses to unisensory conditions). We demonstrate the early-latency, synergistic interplay between existing principles of multisensory interactions. Such findings change the manner in which to model multisensory interactions at neural and behavioral/perceptual levels. We also provide neurophysiologic backing for the notion that looming signals receive preferential treatment during perception.

show abstract

Section: Mechanisms Subserving the Integration Of Looming Signalssupporting

confidence: 63%

Looming Signals Reveal Synergistic Principles of Multisensory Integration

Cappe¹,

Thelen²,

Romei³

et al. 2012

J. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Simultaneously presented sounds can influence visual perception (1, 2) with visual detection or discrimination typically enhanced in the presence of sounds (27)(28)(29). Maximal enhancement occurs when visual stimuli are near the detection/discrimination threshold (28), the auditory and visual stimuli overlap in time and in space (13,29,30) and sounds carry biologically important information (27,29,31).…”

Section: Discussionmentioning

confidence: 99%

Cross-modal effects of value on perceptual acuity and stimulus encoding

Pooresmaeili

FitzGerald

Bach

et al. 2014

Proc. Natl. Acad. Sci. U.S.A.

View full text Add to dashboard Cite

Cross-modal interactions are very common in perception. An important feature of many perceptual stimuli is their reward-predicting properties, the utilization of which is essential for adaptive behavior. What is unknown is whether reward associations in one sensory modality influence perception of stimuli in another modality. Here we show that auditory stimuli with high-reward associations increase the sensitivity of visual perception, even when sounds and reward associations are both irrelevant for the visual task. This increased sensitivity correlates with a change in stimulus representation in the visual cortex, indexed by increased multivariate decoding accuracy in simultaneously acquired functional MRI data. Univariate analysis showed that reward associations modulated responses in regions associated with multisensory processing in which the strength of modulation was a better predictor of the magnitude of the behavioral effect than the modulation in classical reward regions. Our findings demonstrate a value-driven crossmodal interaction that affects perception and stimulus encoding, with a resemblance to well-described modulatory effects of attention. We suggest that multisensory processing areas may mediate the transfer of value signals across senses.reward value | sensory discrimination | audiovisual T he world is structured such that objects or events that cause sensations in one sensory modality influence those in another modality, a mechanism that underlies the near-ubiquitous phenomenon of multisensory interaction (1). This phenomenon has been the focus of a large and burgeoning theoretical and experimental literature (1-4). One feature of environmental stimuli important for adaptive behavior is their rewarding or rewardpredicting properties. Surprisingly, we know little about how reward associations in one sensory modality influence processing in other modlities. In particular, whether an association with increased reward, known to increase the accuracy of perceptual processing within that sensory modality (5-10), can transfer between modalities for concurrently presented stimuli is unclear. This is important both because of what it reveals about the nature of cross-modal associations and because it might constitute an important perceptual mechanism in its own right. For example, one can easily imagine the importance of increased sensitivity to change in the visual scene to the concurrent sound of a tiger roaring or the mating call of a conspecific.To study cross-modal transfer of value, we designed a visual orientation discrimination task in which visual stimuli were presented concurrently with one of two arbitrary pure tones previously associated with different levels of monetary reward. Critically, these tones were task-irrelevant and bore no relationship either to the orientation of the visual stimulus or to the outcome of the trial.(No feedback was presented about the accuracy of perceptual judgments, and performance on the orientation discrimination task was not related to the payment subjects...

show abstract

“…Still others have likewise reported increased coupling between auditory and primary visual cortices, particularly under conditions of synchronous stimulation across the senses (Lewis and Noppeney, 2010;Tyll et al, 2013), that may perhaps be mediated by thalamic circuits (Noesselt et al, 2010;Bonath et al, 2013). In the same vein, sounds have been shown to activate visual cortices as a function of prior multisensory experiences (Zangenehpour and Zatorre, 2010;Meylan and Murray, 2007; see also Murray et al, 2004Murray et al, , 2005Thelen et al, 2012Thelen et al, , 2014 for effects of prior multisensory contexts on sensory processing).…”

Section: Haemodynamic Imagingmentioning

confidence: 94%

The multisensory function of the human primary visual cortex

et al. 2016

View full text Add to dashboard Cite

2It has been nearly ten years since Ghazanfar and Schroeder (2006) proposed that the neocortex is essentially multisensory in nature. However, it is only recently that sufficient and hard evidence that supports this proposal has accrued. We review evidence that activity within the human primary visual cortex plays an active role in multisensory processes and directly impacts behavioural outcome. This evidence emerges from a full pallet of human brain imaging and brain mapping methods with which multisensory processes are quantitatively assessed by taking advantage of particular strengths of each technique as well as advances in signal analyses. Several general conclusions about multisensory processes in primary visual cortex of humans are supported relatively solidly. First, haemodynamic methods (fMRI/PET) show that there is both convergence and integration occurring within primary visual cortex. Second, primary visual cortex is involved in multisensory processes during early post-stimulus stages (as revealed by EEG/ERP/ERFs as well as TMS). Third, multisensory effects in primary visual cortex directly impact behaviour and perception, as revealed by correlational (EEG/ERPs/ERFs) as well as more causal measures (TMS/tACS). While the provocative claim of Ghazanfar and Schroeder (2006) that the whole of neocortex is multisensory in function has yet to be demonstrated, this can now be considered established in the case of the human primary visual cortex.

show abstract

Sound-Induced Enhancement of Low-Intensity Vision: Multisensory Influences on Human Sensory-Specific Cortices and Thalamic Bodies Relate to Perceptual Enhancement of Visual Detection Sensitivity

Cited by 143 publications

References 100 publications

Looming Signals Reveal Synergistic Principles of Multisensory Integration

Looming Signals Reveal Synergistic Principles of Multisensory Integration

Cross-modal effects of value on perceptual acuity and stimulus encoding

The multisensory function of the human primary visual cortex

Contact Info

Product

Resources

About