Hemisphere-specific properties of the ventriloquism aftereffect

Kopčo, Norbert; Lokša, Peter; Lin, I-Fan; Shinn‐Cunningham, Barbara

doi:10.1121/1.5123176

Cited by 6 publications

(36 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A possible explanation of this difference is that the current experiment was performed in a real reverberant room and the subjects were provided with a level-independent auditory distance cue, the direct-to-reverberant energy ratio, whereas in the Gardner study no such cue was available and the subjects likely relied on guessing. The observed constant 72% weight given to the V-component in the current study is much smaller than the weight typically observed in horizontal studies, in which this weight is commonly around 90% (e.g., Kopčo et al 2019).…”

Section: Discussioncontrasting

confidence: 72%

Audio-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Hládek

Seitz

Kopčo

2020

Preprint

Self Cite

View full text Add to dashboard Cite

The processes of audio-visual integration and of visually-guided re-calibration of auditory distance perception are not well understood. Here, the ventriloquism effect (VE) and aftereffect (VAE) were used to study these processes in a real reverberant environment. Auditory and audio-visual (AV) stimuli were presented, in interleaved trials, over a range of distances from 0.7 to 2.04 m in front of the listener, whose task was to judge the distance of auditory stimuli or of the auditory components of AV stimuli. The relative location of the visual and auditory components of AV stimuli was fixed within a session such that the visual component was presented from distance 30% closer (V-closer) than the auditory component, 30% farther (V-farther), or aligned (V-aligned). The study examined the strength of VE and VAE as a function of the reference distance and of the direction of the visual component displacement, and the temporal profile of the build-up/break-down of these effects. All observed effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, measured in the AV trials, was roughly constant for both directions of visual-component displacement such that, on average, the responses shifted in the direction of the visual component by 72% of the audio-visual disparity. The VAE strength, measured on the interleaved auditory-only trials, was stronger in the V-farther than the V-closer condition (44% vs. 31% of the audio-visual disparity, respectively). The VAE persisted to post-adaptation auditory-only blocks of trials, however it was weaker and the V-farther/V-closer asymmetry was reduced. The rates of build-up/break-down of the VAE were also asymmetrical, with slower adaptation in the V-closer condition. These results suggest that, on a logarithmic scale, the AV distance integration is symmetrical, independent of the direction of induced shift, while the visually-induced auditory distance re-callibration is asymmetrical, stronger and faster when evoked by more distant visual stimuli.

show abstract

Section: Discussioncontrasting

confidence: 72%

Audio-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Hládek

Seitz

Kopčo

2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…A means of aligning these RFs is necessary by the stage at which the visual signals guide auditory spatial adaptation. Our previous studies suggest that a mixture of eye-centered and head-centered RFs is associated with recalibration in the central region of the audiovisual field [12] while the head-centered RF dominates for VAE locally induced in a single hemifield in the visual periphery [13].…”

Section: Introductionmentioning

confidence: 96%

“…Finally, Kopco et al [13] observed a new adaptive phenomenon induced by aligned audiovisual stimuli presented in the periphery, exhibited as a shift in responses to sounds Loksa presented alone in the central region. The shift magnitude depended on the gaze direction and, thus, was at least partly in the eye-centered RF.…”

Section: Introductionmentioning

confidence: 98%

“…It assumes that the RF transformations are the same across the audio-visual field, and that the observed location-dependence is due to other adaptive processes, e.g., related to auditory saccade adaptation, as saccades were used to measure behavioral responses in the Kopco et al [12,13] studies. The main modeling goal is then to determine whether such a uniform, location-independent spatial adaptation is only driven by head-orientation referenced visual signals, or whether signals in eye-centered RF also contribute.…”

Section: Introductionmentioning

confidence: 99%

“…Previous studies show that auditory saccades can overestimate or underestimate the actual sound locations [16] and that the amount of visually induced adaptation does not depend on whether the resulting saccades are hypometric or hypermetric [17]. Here, in the Appendix, we analyze the data from Kopco et al [12,13] to determine whether the ventriloquism effect and aftereffect show asymmetries depending on the resulting adaptation type (hypometric vs. hypermetric), as well as on the saccade amplitude magnitude. Based on this analysis, the current model assumes that the magnitude of the ventriloquism aftereffect is proportional to the magnitude of the ventriloquism effect, independent of whether these shifts result in hypometric or hypermetric saccades, and independent on the saccade magnitude.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Towards a unified theory of the reference frame of the ventriloquism aftereffect

Lokša

Kopčo²

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Background: Ventriloquism aftereffect (VAE), observed as a shift in the perceived locations of sounds after audiovisual stimulation, requires reference frame (RF) alignment since hearing and vision encode space in different RFs (head-centered, HC, vs. eye-centered, EC). Experimental studies examining the RF of VAE found inconsistent results: a mixture of HC and EC RFs was observed for VAE induced in the central region, while a predominantly HC RF was observed in the periphery. Here, a computational model examines these inconsistencies, as well as a newly observed EC adaptation induced by AV-aligned audiovisual stimuli. Methods: The model has two versions, each containing two additively combined components: a saccade-related component characterizing the adaptation in auditory-saccade responses, and auditory space representation adapted by ventriloquism signals either in the HC RF (HC version) or in a combination of HC and EC RFs (HEC version). Results: The HEC model performed better than the HC model in the main simulation considering all the data, while the HC model was more appropriate when only the AV-aligned adaptation data were simulated. Conclusion: Visual signals in a uniform mixed HC+EC RF are likely used to calibrate the auditory spatial representation, even after the EC-referenced auditory-saccade adaptation is accounted for.

show abstract

Auditory-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Hládek

Seitz

Kopčo

2021

The Journal of the Acoustical Society of America

View full text Add to dashboard Cite

This study describes data on auditory-visual integration and visually-guided adaptation of auditory distance perception using the ventriloquism effect (VE) and ventriloquism aftereffect (VAE). In an experiment, participants judged egocentric distance of interleaved auditory or auditory-visual stimuli with the auditory component located from 0.7 to 2.04 m in front of listeners in a real reverberant environment. The visual component of auditory-visual stimuli was displaced 30% closer (V-closer), 30% farther (V-farther), or aligned (V-aligned) with respect to the auditory component. The VE and VAE were measured in auditory and auditory-visual trials, respectively. Both effects were approximately independent of target distance when expressed in logarithmic units. The VE strength, defined as a difference of V-misaligned and V-aligned response bias, was approximately 72% of the auditory-visual disparity regardless of the visual-displacement direction, while the VAE was stronger in the V-farther (44%) than the V-closer (31%) condition. The VAE persisted to post-adaptation auditory-only blocks of trials, although it was diminished. The rates of build-up/break-down of the VAE were asymmetrical, with slower adaptation in the V-closer condition. These results suggest that auditory-visual distance integration is independent of the direction of induced shift, while the re-calibration is stronger and faster when evoked by more distant visual stimuli.

show abstract

Hemisphere-specific properties of the ventriloquism aftereffect

Cited by 6 publications

References 15 publications

Audio-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Audio-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Towards a unified theory of the reference frame of the ventriloquism aftereffect

Auditory-visual interactions in egocentric distance perception: Ventriloquism effect and aftereffect

Contact Info

Product

Resources

About