The spread of attention across modalities and space in a multisensory object

Abstract

Attending to a stimulus is known to enhance the neural responses to that stimulus. Recent experiments on visual attention have shown that this modulation can have object-based characteristics, such that, when certain parts of a visual object are attended, other parts automatically also receive enhanced processing. Here, we investigated whether visual attention can modulate neural responses to other components of a multisensory object defined by synchronous, but spatially disparate, auditory and visual stimuli. The audiovisual integration of such multisensory stimuli typically leads to mislocalization of the sound toward the visual stimulus (ventriloquism illusion). Using event-related potentials and functional MRI, we found that the brain's response to task-irrelevant sounds occurring synchronously with a visual stimulus from a different location was larger when that accompanying visual stimulus was attended versus unattended. The event-related potential effect consisted of sustained, frontally distributed, brain activity that emerged relatively late in processing, an effect resembling attention-related enhancements seen at earlier latencies during intramodal auditory attention. Moreover, the functional MRI data confirmed that the effect included specific enhancement of activity in auditory cortex. These findings indicate that attention to one sensory modality can spread to encompass simultaneous signals from another modality, even when they are task-irrelevant and from a different location. This cross-modal attentional spread appears to reflect an object-based, late selection process wherein spatially discrepant auditory stimulation is grouped with synchronous attended visual input into a multisensory object, resulting in the auditory information being pulled into the attentional spotlight and bestowed with enhanced processing.A ttention allows us to dynamically select and enhance the processing of stimuli and events that are most relevant at each moment. Directing attention to a stimulus leads to lower perceptual thresholds, faster reaction times (RTs), and increased discrimination accuracy (1-2). The physiological basis for these perceptual benefits involves enhanced neural activity in response to the attended stimulus, as has been shown by a variety of brain activity measures (e.g., refs. 3-5).Attention can be directed to a spatial location (2-3, 6-7), enhancing the processing of all stimuli occurring at that location, or to a particular stimulus feature (e.g., the color red, a certain tonal frequency), resulting in preferential processing of that feature independent of its spatial location (8-9). Furthermore, it has been proposed that attention can also act on whole objects, such that when attention is directed to one part of an object, the other components of the same object automatically receive enhanced processing (10-12).So far, studies of object-based attentional selection have focused on the visual modality. In these experiments, subjects are typically cued to direct attention to a spatially defined p...

The neural circuitry underlying the executive control of auditory spatial attention

“…A second possibility is suggested by neuroanatomical and neurophysiological studies, which have revealed that the striate cortex receives neuronal projections from auditory cortices, which may facilitate multimodal integration (Clavagnier et al, 2004;Falchier et al, 2002;Rockland and Ojima, 2003;Schroeder and Foxe, 2005). We speculate that these anatomically early connections might allow attentional modulations of sensory activity in one modality to "spread" to sensory cortices in a second modality under certain conditions (Busse et al, 2005). Additional studies are clearly needed to determine the precise nature of the cross-modal attentional effects that we have observed here.…”

Topographic maps of multisensory attention

“…For example, a visual stimulus can enhance attention to an auditory stimulus localized to the same region of space (31). Interactions between sensory modalities can occur either due to connections between primary sensory regions or through multisensory convergence zones, and can result in competitive, additive, or superadditive effects of sensory stimulation (32).…”

Object-based auditory and visual attention

“…Most work on attention and objects is in the visual literature [2,21], but similar principles govern auditory perception [22,23,24]. Evidence suggests that attention acts on auditory objects, much as it enhances visual objects [25,26,27].…”