Audio-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Abstract

The processes of audio-visual integration and of visually-guided re-calibration of auditory distance perception are not well understood. Here, the ventriloquism effect (VE) and aftereffect (VAE) were used to study these processes in a real reverberant environment. Auditory and audio-visual (AV) stimuli were presented, in interleaved trials, over a range of distances from 0.7 to 2.04 m in front of the listener, whose task was to judge the distance of auditory stimuli or of the auditory components of AV stimuli. The relative location of the visual and auditory components of AV stimuli was fixed within a session such that the visual component was presented from distance 30% closer (V-closer) than the auditory component, 30% farther (V-farther), or aligned (V-aligned). The study examined the strength of VE and VAE as a function of the reference distance and of the direction of the visual component displacement, and the temporal profile of the build-up/break-down of these effects. All observed effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, measured in the AV trials, was roughly constant for both directions of visual-component displacement such that, on average, the responses shifted in the direction of the visual component by 72% of the audio-visual disparity. The VAE strength, measured on the interleaved auditory-only trials, was stronger in the V-farther than the V-closer condition (44% vs. 31% of the audio-visual disparity, respectively). The VAE persisted to post-adaptation auditory-only blocks of trials, however it was weaker and the V-farther/V-closer asymmetry was reduced. The rates of build-up/break-down of the VAE were also asymmetrical, with slower adaptation in the V-closer condition. These results suggest that, on a logarithmic scale, the AV distance integration is symmetrical, independent of the direction of induced shift, while the visually-induced auditory distance re-callibration is asymmetrical, stronger and faster when evoked by more distant visual stimuli.

Perceptual Matching of Room Acoustics for Auditory Augmented Reality in Small Rooms - Literature Review and Theoretical Framework

“…The visual distance to an object representing the sound source also affects the auditory distance perception and dominates the audio-visual distance judgment (Medonça et al, 2016). A more recent study by Hládek et al (2020) investigated the VE and VAE for auditory distances between 0.7 and 2.03 m for fixed 30% relative shifts of the visual component. The study was conducted in a small semi-reverberant room with 183 participants.…”