Integration of visual information in auditory cortex promotes auditory scene analysis through multisensory binding

Atilgan, Huriye; Town, Stephen M.; Wood, Katherine C.; Jones, Gareth P.; Maddox, Ross K.; Lee, Adrian K. C.; Bizley, Jennifer K.

doi:10.1101/098798

Cited by 25 publications

(41 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This effect is principally driven by an improvement in the ability of listeners to exploit 26 temporal coherence in the masker--coherent condition. We have demonstrated that the enhancement of 27 one sound in a mixture by a temporally coherent visual stimulus is a stimulus driven, attention 28 independent, bottom--up effect supported by the early integration of auditory and visual information in 29 auditory cortex (Atilgan et al, 2018). In keeping with our behavioral data from naïve listeners, such an 1 enhancement seems likely to facilitate selective attention when the temporally coherent stream is a 2 target, and impair it when that sound is a distractor.…”

Section: Discussion 23supporting

confidence: 63%

Training enhances the ability of listeners to exploit visual information for auditory scene analysis

Atilgan

Bizley

2018

Preprint

Self Cite

View full text Add to dashboard Cite

1The ability to use temporal relationships between cross-modal cues facilitates perception and 2 behavior. Previously we observed that temporally correlated changes in the size of a visual 3 stimulus and the intensity in an auditory stimulus influenced the ability of listeners to perform an 4 auditory selective attention task (Maddox et al., 2015). In this task participants detected timbral 5 changes in a target sound while ignoring those in a simultaneously presented masker. When the 6 visual stimulus was temporally coherent with the target sound, performance was significantly 7 better than when it was temporally coherent with the masker sound, despite the visual stimulus 8 conveying no task-relevant information. Here, we trained observers to detect audiovisual 9 temporal coherence and asked whether this improved their ability to benefit from visual cues 10 during the auditory selective attention task. We observed these listeners improved performance 11 in the auditory selective attention task and changed the way in which they benefited from a 12 visual stimulus: after training performance was better when the visual stimulus was temporally 13 coherent with either the target or the masker stream, relative to the condition in which the visual 14 stimulus was coherent with neither auditory stream. A second group which trained to 15 discriminate modulation rate differences between temporally coherent audiovisual streams 16 improved task performance, but did not change the way in which they used visual information. A 17 control group did not change their performance between pretest and post-test. These results 18 provide insights into how crossmodal experience may optimize multisensory integration. 19 20 Keywords: audiovisual integration, selective attention, auditory scene analysis, temporal 21 processing, training 22 A problem in interpreting these varied effects is that many lab-based tasks do not well capture 2 the complexity that the brain faces in real-world situations. In most lab-based paradigms 3 observers often judge single audio and visual signals, presented in an otherwise quiet and dark 4 environment. In contrast, in the world, brain must match one of several competing sounds to a 5 given visual object (or vice versa). Moreover, due to the variance in the timing of real-world 6 temporal coherence of auditory and visual streams. Prior to training, participants showed the expected effect of coherence condition (i.e. only target coherent > masker coherent). However,

show abstract

Section: Discussion 23supporting

confidence: 63%

Training enhances the ability of listeners to exploit visual information for auditory scene analysis

Atilgan

Bizley

2018

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…To simulate the effects of cooling on responses of individual units, we built a simple model mapping suppression of spiking activity to reduction in cortical temperature using data obtained from simultaneous cooling and single unit recording in Suprasylvian cortex (SSY; higher order visual cortex) (Wood, Town et al 2017, Atilgan, Town et al 2018. Figure 3B illustrates the suppression of responses of one SSY unit to visual stimuli when cooling was applied.…”

Section: Population Decoding Offers a Mechanism For Effects Of Coolingmentioning

confidence: 99%

“…Neural activity was recorded with 16-channel, single shank silicon probe (A1x16, Neuronexus Technologies, Ann Arbor, MI) placed at the center of a cooling loop, which was itself placed on the suprasylvian cortex, in the region posterior to auditory cortex (See Atilgan, Town et al 2018 for further details). The design of the cooling loop system was similar to that designed for behaving experiments, with the exception that the loop was not embedded within dental cement during anesthetized experiments.…”

Section: Neural Recording During Coolingmentioning

confidence: 99%

“…Following a period where the electrode was left alone to ensure stability of recording (5 -30 minutes), ferrets were presented with 100 ms stimuli that were either auditory (broadband noise), visual (white LED) and audiovisual stimuli (noise + LED) (Atilgan, Town et al 2018). Each stimulus was repeated 20 -40 times in a random order with a jittered interval between presentations (minimum: 900 ms).…”

Section: Neural Recording During Coolingmentioning

confidence: 99%

See 1 more Smart Citation

Signal processing in auditory cortex underlies degraded speech sound discrimination in noise

Town

Wood

Bizley

2019

Preprint

Self Cite

View full text Add to dashboard Cite

The ability to recognize sounds in noise is a key part of hearing, and the mechanisms by which the brain identifies sounds in noise are of considerable interest to scientists, clinicians and engineers. Yet we know little about the necessity of regions such as auditory cortex for hearing in noise, or how cortical processing of sounds is adversely affected by noise. Here we used reversible cortical inactivation and extracellular electrophysiology in ferrets performing a vowel discrimination task to identify and understand the causal contribution of auditory cortex to hearing in noise. Cortical inactivation by cooling impaired task performance in noisy but not clean conditions, while responses of auditory cortical neurons were less informative about vowel identity in noise. Simulations mimicking cortical inactivation indicated that effects of inactivation were related to the loss of information about sounds represented across neural populations. The addition of noise to target sounds drove spiking activity in auditory cortex and recruitment of additional neural populations that were linked to degraded behavioral performance. To suppress noise-related activity, we used continuous exposure to background noise to adapt the auditory system and recover behavioral performance in both ferrets and humans. Inactivation by cooling revealed that the benefits of continuous exposure were not cortically dependent. Together our results highlight the importance of auditory cortex in sound discrimination in noise and the underlying mechanisms through which noise-related activity and adaptation shape hearing.

show abstract

“…The neural basis of multisensory integration and its loci in the hierarchy of brain 5 computations have been the focus of myriad of studies (see for reviews (Talsma et al, 2010;ten 6 Oever et al, 2016;Keil and Senkowski, 2018)). It is now well established that multisensory 7 integration starts early in the process chain Schroeder and Foxe, 8 2005): both animal and human studies have demonstrated that the genesis of multisensory 9 integration relies on cross-modal inputs to sensory cortices which informs about the 10 spatiotemporal co-occurrence of sensory cues (Bizley et al, 2007;Lakatos et al, 2007;Kayser 11 et al, 2008;Cappe et al, 2010;Mercier et al, 2013Mercier et al, , 2015Atilgan et al, 2018). At a later stage, 12 the integration of information from different modalities pertains to congruency and reliability of 13 multisensory inputs, as well as task relevance Noppeney, 2015, 2016;Kayser et al, 14 2017).…”

mentioning

confidence: 99%

The interplay between multisensory integration and perceptual decision making

Mercier

Cappe

2019

Preprint

View full text Add to dashboard Cite

2Facing perceptual uncertainty, the brain combines information from different senses to shape 3 optimal decision making and to guide behavior. Despite overlapping neural networks underlying 4 multisensory integration and perceptual decision making, the process chain of decision formation 5 has been studied mostly in unimodal contexts and is thought to be supramodal. To reveal whether 6 and how multisensory processing interplay with perceptual decision making, we devised a 7 paradigm mimicking naturalistic situations where human participants were exposed to continuous 8 cacophonous audiovisual inputs containing an unpredictable relevant signal cue in one or two 9 modalities. Using multivariate pattern analysis on concurrently recorded EEG, we decoded the 10 neural signatures of sensory encoding and decision formation stages. Generalization analyses 11 across conditions and time revealed that multisensory signal cues were processed faster during 12 both processing stages. We further established that acceleration of neural dynamics was directly 13 linked to two distinct multisensory integration processes and associated with multisensory benefit. 14 Our results, substantiated in both detection and categorization tasks, provide evidence that the 15 brain integrates signals from different modalities at both the sensory encoding and the decision 16 formation stages. 17

show abstract

Integration of visual information in auditory cortex promotes auditory scene analysis through multisensory binding

Cited by 25 publications

References 58 publications

Training enhances the ability of listeners to exploit visual information for auditory scene analysis

Training enhances the ability of listeners to exploit visual information for auditory scene analysis

Signal processing in auditory cortex underlies degraded speech sound discrimination in noise

The interplay between multisensory integration and perceptual decision making

Contact Info

Product

Resources

About