Relative Unisensory Strength and Timing Predict Their Multisensory Product

Miller, Ryan; Pluta, Scott R.; Stein, Barry E.; Rowland, Benjamin A.

doi:10.1523/jneurosci.4771-14.2015

Cited by 39 publications

(55 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For example, as shown in the present study (Experiment 1), the rat V2L cortex—a well-established area responsive to audiovisual stimuli (Toldi et al, 1986; Barth et al, 1995; Wallace et al, 2004; Hirokawa et al, 2008; Xu et al, 2014; Schormans et al, 2016)—is sensitive to differences in the timing of combined audiovisual stimuli, such that spiking activity was greatest during trials when the visual stimulus preceded the auditory by 20–40 ms (Figures 7C, 8C). These results are fairly consistent with previous studies that recorded audiovisual-evoked spiking activity in the superior colliculus (cat (Meredith and Stein, 1986, 1996; Meredith et al, 1987; Perrault et al, 2005, 2012; Stanford et al, 2005) and guinea pig (King and Palmer, 1985)) as well as multisensory cortices (cat PLLS (Allman and Meredith, 2007; Allman et al, 2008b, 2009) and cat FAES (Meredith and Allman, 2009)), and further confirm that the timing of the stimuli play a critical role in the ability of the neurons to integrate the different sensory modalities (King and Palmer, 1985; Meredith and Stein, 1986; Perrault et al, 2005; Stanford et al, 2005; Miller et al, 2015). Although the V2L cortex has been shown to play an important role in audiovisual processing, future investigations are needed in order to assess audiovisual temporal processing at the single neuron level.…”

Section: Discussionmentioning

confidence: 69%

Audiovisual Temporal Processing and Synchrony Perception in the Rat

Schormans

Scott

et al. 2017

Front. Behav. Neurosci.

View full text Add to dashboard Cite

Extensive research on humans has improved our understanding of how the brain integrates information from our different senses, and has begun to uncover the brain regions and large-scale neural activity that contributes to an observer’s ability to perceive the relative timing of auditory and visual stimuli. In the present study, we developed the first behavioral tasks to assess the perception of audiovisual temporal synchrony in rats. Modeled after the parameters used in human studies, separate groups of rats were trained to perform: (1) a simultaneity judgment task in which they reported whether audiovisual stimuli at various stimulus onset asynchronies (SOAs) were presented simultaneously or not; and (2) a temporal order judgment task in which they reported whether they perceived the auditory or visual stimulus to have been presented first. Furthermore, using in vivo electrophysiological recordings in the lateral extrastriate visual (V2L) cortex of anesthetized rats, we performed the first investigation of how neurons in the rat multisensory cortex integrate audiovisual stimuli presented at different SOAs. As predicted, rats (n = 7) trained to perform the simultaneity judgment task could accurately (~80%) identify synchronous vs. asynchronous (200 ms SOA) trials. Moreover, the rats judged trials at 10 ms SOA to be synchronous, whereas the majority (~70%) of trials at 100 ms SOA were perceived to be asynchronous. During the temporal order judgment task, rats (n = 7) perceived the synchronous audiovisual stimuli to be “visual first” for ~52% of the trials, and calculation of the smallest timing interval between the auditory and visual stimuli that could be detected in each rat (i.e., the just noticeable difference (JND)) ranged from 77 ms to 122 ms. Neurons in the rat V2L cortex were sensitive to the timing of audiovisual stimuli, such that spiking activity was greatest during trials when the visual stimulus preceded the auditory by 20–40 ms. Ultimately, given that our behavioral and electrophysiological results were consistent with studies conducted on human participants and previous recordings made in multisensory brain regions of different species, we suggest that the rat represents an effective model for studying audiovisual temporal synchrony at both the neuronal and perceptual level.

show abstract

Section: Discussionmentioning

confidence: 69%

Audiovisual Temporal Processing and Synchrony Perception in the Rat

Schormans

Scott

et al. 2017

Front. Behav. Neurosci.

View full text Add to dashboard Cite

show abstract

“…Many electrophysiological studies have already shown that multisensory neurons exhibit an integrating window and SOAs of different stimuli can fall into this window and become well integrated as a single response by multisensory neurons (Miller et al . ; Felch et al . ).…”

Section: Discussionmentioning

confidence: 99%

Spatial receptive field shift by preceding cross‐modal stimulation in the cat superior colliculus

et al. 2018

The Journal of Physiology

View full text Add to dashboard Cite

Psychophysical studies have shown that the different senses can be spatially entrained by each other. This can be observed in certain phenomena, such as ventriloquism, in which a visual stimulus can attract the perceived location of a spatially discordant sound. However, the neural mechanism underlying this cross-modal spatial recalibration has remained unclear, as has whether it takes place dynamically. We explored these issues in multisensory neurons of the cat superior colliculus (SC), a midbrain structure that involves both cross-modal and sensorimotor integration. Sequential cross-modal stimulation showed that the preceding stimulus can shift the receptive field (RF) of the lagging response. This cross-modal spatial calibration took place in both auditory and visual RFs, although auditory RFs shifted slightly more. By contrast, if a preceding stimulus was from the same modality, it failed to induce a similarly substantial RF shift. The extent of the RF shift was dependent on both temporal and spatial gaps between the preceding and following stimuli, as well as the salience of the preceding stimulus. A narrow time gap and high stimulus salience were able to induce larger RF shifts. In addition, when both visual and auditory stimuli were presented simultaneously, a substantial RF shift toward the location-fixed stimulus was also induced. These results, taken together, reveal an online cross-modal process and reflect the details of the organization of SC inter-sensory spatial calibration.

show abstract

“…Instead, it is supported by multimodal neurons at some of the earliest processing stages, including the superior colliculus (SC) [79,80] and the thalamus [81], and increases in strength along cortical pathways [40]. In more detail, the SC, a midbrain structure, divides into superficial layers that are exclusively visual, and deeper layers that combine visual, auditory, and somatosensory input.…”

Section: Modality Similarities Differences and Convergencementioning

confidence: 99%

“…In more detail, the SC, a midbrain structure, divides into superficial layers that are exclusively visual, and deeper layers that combine visual, auditory, and somatosensory input. Reciprocal connections between these layers and their onward projections to the thalamus likely support early aspects of multimodal processing such as temporal binding and cross-modal enhancement (e.g., reducing a unimodal detection threshold) [79,80]. …”

Section: Modality Similarities Differences and Convergencementioning

confidence: 99%

Emotion Perception from Face, Voice, and Touch: Comparisons and Convergence

Schirmer

Adolphs

2017

Trends in Cognitive Sciences

308

189

View full text Add to dashboard Cite

Historically, research on emotion perception has focused on facial expressions, and findings from this modality have come to dominate our thinking about other modalities. Here, we examine emotion perception through a wider lens by comparing facial with vocal and tactile processing. We review stimulus characteristics and ensuing behavioral and brain responses, and show that audition and touch do not simply duplicate visual mechanisms. Each modality provides a distinct input channel and engages partly non-overlapping neuroanatomical systems with different processing specializations (e.g., specific emotions versus affect). Moreover, processing of signals across the different modalities converges, first into multi- and later into amodal representations that enable holistic emotion judgments.

show abstract

Relative Unisensory Strength and Timing Predict Their Multisensory Product

Cited by 39 publications

References 31 publications

Audiovisual Temporal Processing and Synchrony Perception in the Rat

Audiovisual Temporal Processing and Synchrony Perception in the Rat

Spatial receptive field shift by preceding cross‐modal stimulation in the cat superior colliculus

Emotion Perception from Face, Voice, and Touch: Comparisons and Convergence

Contact Info

Product

Resources

About