Visual Localization Ability Influences Cross-Modal Bias

Abstract

The ability of a visual signal to influence the localization of an auditory target (i.e., "cross-modal bias") was examined as a function of the spatial disparity between the two stimuli and their absolute locations in space. Three experimental issues were examined: (a) the effect of a spatially disparate visual stimulus on auditory localization judgments; (b) how the ability to localize visual, auditory, and spatially aligned multisensory (visual-auditory) targets is related to cross-modal bias, and (c) the relationship between the magnitude of cross-modal bias and the perception that the two stimuli are spatially "unified" (i.e., originate from the same location). Whereas variability in localization of auditory targets was large and fairly uniform for all tested locations, variability in localizing visual or spatially aligned multisensory targets was much smaller, and increased with increasing distance from the midline. This trend proved to be strongly correlated with biasing effectiveness, for although visual-auditory bias was unexpectedly large in all conditions tested, it decreased progressively (as localization variability increased) with increasing distance from the midline. Thus, central visual stimuli had a substantially greater biasing effect on auditory target localization than did more peripheral visual stimuli. It was also apparent that cross-modal bias decreased as the degree of visual-auditory disparity increased. Consequently, the greatest visual-auditory biases were obtained with small disparities at central locations. In all cases, the magnitude of these biases covaried with judgments of spatial unity. The results suggest that functional properties of the visual system play the predominant role in determining these visual-auditory interactions and that cross-modal biases can be substantially greater than previously noted.

Generative Probabilistic Modeling: Understanding Causal Sensorimotor Integration

“…The second key aspect of this task which must be included in any full generative model of this problem is that oddity can be entailed in the probe stimulus not only by its combined difference from the standard, but by discrepancy within the probe stimulus. In this case (similar to other recent multisensory perception experiments with variable causal structure : Hairston et al, 2003;Shams et al, 2000Shams et al, , 2005Wallace et al, 2004), the variable structure can effectively "give away" the probe. We introduced the approach needed to solve this type of problem in multisensory perception as structure inference (Hospedales & Vijayakumar, 2008).…”

“…That is, it was not obvious which of multiple possible sources caused the observations (Hairston et al, 2003;Shams, Kamitani, & Shimojo, 2000;Shams, Ma, & Beierholm, 2005;Wallace et al, 2004), or which of multiple possible world models was true (Knill, 2007). In these cases, the standard MLI linear-cue-combination approach fails to explain human performance.…”

“…We introduced the approach needed to solve this type of problem in multisensory perception as structure inference (Hospedales & Vijayakumar, 2008). Körding, Beierholm et al (2007) carried out a detailed analysis of the experiments of Hairston et al (2003) and Wallace et al (2004) and showed how the structure inference approach was necessary to explain the results, but they termed the procedure causal inference (see Chapter 2).…”

Intersensory binding across space and time: A tutorial review

“…From the literature, it appears that they are quite different. For spatial ventriloquism, several behavioral and physiological studies have shown that the spatial ventriloquist effect disappears when the audiovisual temporal alignment is outside a −100 -+ 300 ms window (−100 ms = sound before vision; +300 ms=sound after vision), while the horizontal spatial alignment should not exceed~15° (Godfroy et al, 2003;Hairston, Wallace, Vaughan, Stein, Norris & Schirillo, 2003;Lewald & Guski, 2003;Slutsky & Recanzone, 2001;Radeau & Bertelson, 1977), although the specific degree of tolerated disparities could take a wide range (Wallace, Roberson, Hairston, Stein, Vaughan & Schirillo, 2004).…”

Auditory–visual temporal integration measured by shifts in perceived temporal location

“…These experiments have established 'spatial ventriloquism' [11] in which the perceived location of the auditory component of an auditory/visual multisensory stimulus is displaced in a statistically optimal way towards the component with the least variability in localization estimates [1,9]. The dominance of vision in ventriloquism is compatible with vision usually being the least variable sense in specifying spatial location [22,33].…”