Visual search for a target changing in synchrony with an auditory signal

Fujisaki, Waka; Koene, Ansgar; Arnold, Derek H.; Johnston, Alan; Nishida, Shin ' ya

doi:10.1098/rspb.2005.3327

Cited by 77 publications

(84 citation statements)

References 49 publications

(77 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This could help reduce effective set size, and thus perceptual load, allowing audio-visual integration to be more effective. This explanation is indirectly supported by previous findings indicating that cross-modal integration under highperceptual-load conditions is mediated by a serial, attentive process [38,39,52], and therefore should be more effective in conditions where there are fewer possible auditory-visual associations. Audio-visual coincidence selection can be enabled in a variety of ways, such as using sparse visual displays (as in many multi-sensory enhancement experiments), or by the saliency and temporal informativeness of the accessory acoustic cue [36].…”

Section: Discussionsupporting

confidence: 73%

See 1 more Smart Citation

Acoustic facilitation of object movement detection during self-motion

Calabro¹,

Soto‐Faraco

Vaina

2011

Proc. R. Soc. B.

View full text Add to dashboard Cite

In humans, as well as most animal species, perception of object motion is critical to successful interaction with the surrounding environment. Yet, as the observer also moves, the retinal projections of the various motion components add to each other and extracting accurate object motion becomes computationally challenging. Recent psychophysical studies have demonstrated that observers use a flow-parsing mechanism to estimate and subtract self-motion from the optic flow field. We investigated whether concurrent acoustic cues for motion can facilitate visual flow parsing, thereby enhancing the detection of moving objects during simulated self-motion. Participants identified an object (the target) that moved either forward or backward within a visual scene containing nine identical textured objects simulating forward observer translation. We found that spatially co-localized, directionally congruent, moving auditory stimuli enhanced object motion detection. Interestingly, subjects who performed poorly on the visualonly task benefited more from the addition of moving auditory stimuli. When auditory stimuli were not co-localized to the visual target, improvements in detection rates were weak. Taken together, these results suggest that parsing object motion from self-motion-induced optic flow can operate on multisensory object representations.

show abstract

Section: Discussionsupporting

confidence: 73%

“…Moreover, paradigms where perceptual load is high (i.e. when the matching between sound and visual events must be extracted from complex, dynamically changing events) have typically failed to demonstrate cross-modal enhancement in search tasks [38,39].…”

Section: Introductionmentioning

confidence: 99%

Acoustic facilitation of object movement detection during self-motion

Calabro¹,

Soto‐Faraco

Vaina

2011

Proc. R. Soc. B.

View full text Add to dashboard Cite

show abstract

“…This may be critical to the low-level interactions between the two modalities reported by these studies. Many studies have reported that attention is a prerequisite for observing multisensory integration (Alsius, Navarra, Campbell, & Soto-Faraco, 2005;Alsius, Navarra, & Soto-Faraco, 2007;Fujisaki, Koene, Arnold, Johnston, & Nishida, 2006;Talsma, Doty, & Woldorff, 2007), although others report findings to the contrary ( Van der Burg et al, 2008a;and see Talsma, Senkowski, Soto-Faraco, & Woldorff, 2010, for a recent review regarding the role of attention in multisensory integration).…”

Section: Discussionmentioning

confidence: 99%

Cross-modal associations between vision, touch, and audition influence visual search through top-down attention, not bottom-up capture

Orchard-Mills

Alais

Burg

2013

Atten Percept Psychophys

View full text Add to dashboard Cite

Recently, Guzman-Martinez, Ortega, Grabowecky, Mossbridge, and Suzuki (Current Biology : CB, 22(5), [383][384][385][386][387][388] 2012) reported that observers could systematically match auditory amplitude modulations and tactile amplitude modulations to visual spatial frequencies, proposing that these cross-modal matches produced automatic attentional effects. Using a series of visual search tasks, we investigated whether informative auditory, tactile, or bimodal cues can guide attention toward a visual Gabor of matched spatial frequency (among others with different spatial frequencies). These cues improved visual search for some but not all frequencies. Auditory cues improved search only for the lowest and highest spatial frequencies, whereas tactile cues were more effective and frequency specific, although less effective than visual cues. Importantly, although tactile cues could produce efficient search when informative, they had no effect when uninformative. This suggests that cross-modal frequency matching occurs at a cognitive rather than sensory level and, therefore, influences visual search through voluntary, goaldirected behavior, rather than automatic attentional capture.

show abstract

“…The mean 4 Hz stimulation rate (range of 2-10) used here, together with the constraints (protective window, see Fig. 1 B) implemented to avoid any accidental synchronies in the noncorresponding condition, should optimize detection of audiovisual correspondence versus noncorrespondence (Fujisaki et al, 2006) but make these bimodal conditions otherwise identical in terms of the temporal patterns presented overall to each modality. All sequences were created individually for each subject using Matlab 6.5 (MathWorks, Natick, MA).…”

Section: Methodsmentioning

confidence: 99%

Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

Noesselt

Rieger²,

Schoenfeld³

et al. 2007

J. Neurosci.

264

235

View full text Add to dashboard Cite

The brain should integrate related but not unrelated information from different senses. Temporal patterning of inputs to different modalities may provide critical information about whether those inputs are related or not. We studied effects of temporal correspondence between auditory and visual streams on human brain activity with functional magnetic resonance imaging (fMRI). Streams of visual flashes with irregularly jittered, arrhythmic timing could appear on right or left, with or without a stream of auditory tones that coincided perfectly when present (highly unlikely by chance), were noncoincident with vision (different erratic, arrhythmic pattern with same temporal statistics), or an auditory stream appeared alone. fMRI revealed blood oxygenation level-dependent (BOLD) increases in multisensory superior temporal sulcus (mSTS), contralateral to a visual stream when coincident with an auditory stream, and BOLD decreases for noncoincidence relative to unisensory baselines. Contralateral primary visual cortex and auditory cortex were also affected by audiovisual temporal correspondence or noncorrespondence, as confirmed in individuals. Connectivity analyses indicated enhanced influence from mSTS on primary sensory areas, rather than vice versa, during audiovisual correspondence. Temporal correspondence between auditory and visual streams affects a network of both multisensory (mSTS) and sensory-specific areas in humans, including even primary visual and auditory cortex, with stronger responses for corresponding and thus related audiovisual inputs.

show abstract

Visual search for a target changing in synchrony with an auditory signal

Cited by 77 publications

References 49 publications

Acoustic facilitation of object movement detection during self-motion

Acoustic facilitation of object movement detection during self-motion

Cross-modal associations between vision, touch, and audition influence visual search through top-down attention, not bottom-up capture

Audiovisual Temporal Correspondence Modulates Human Multisensory Superior Temporal Sulcus Plus Primary Sensory Cortices

Contact Info

Product

Resources

About