Ambisonics Sound Source Localization With Varying Amount of Visual Information in Virtual Reality

Abstract

To reproduce realistic audio-visual scenarios in the laboratory, Ambisonics is often used to reproduce a sound field over loudspeakers and virtual reality (VR) glasses are used to present visual information. Both technologies have been shown to be suitable for research. However, the combination of both technologies, Ambisonics and VR glasses, might affect the spatial cues for auditory localization and thus, the localization percept. Here, we investigated how VR glasses affect the localization of virtual sound sources on the horizontal plane produced using either 1st-, 3rd-, 5th- or 11th-order Ambisonics with and without visual information. Results showed that with 1st-order Ambisonics the localization error is larger than with the higher orders, while the differences across the higher orders were small. The physical presence of the VR glasses without visual information increased the perceived lateralization of the auditory stimuli by on average about 2°, especially in the right hemisphere. Presenting visual information about the environment and potential sound sources did reduce this HMD-induced shift, however it could not fully compensate for it. While the localization performance itself was affected by the Ambisonics order, there was no interaction between the Ambisonics order and the effect of the HMD. Thus, the presence of VR glasses can alter acoustic localization when using Ambisonics sound reproduction, but visual information can compensate for most of the effects. As such, most use cases for VR will be unaffected by these shifts in the perceived location of the auditory stimuli.

Spatial Resolution of Late Reverberation in Virtual Acoustic Environments

“…While artificial reverberation has historically been used as a tool for artistic expression (Välimäki et al, 2012), room acoustics simulations are also utilized for planning of spaces (e.g., concert halls and classrooms; Rindel, 2001). Recently, room acoustics simulations and virtual acoustic environments (VAEs) have gained interest as tools for psychoacoustic research and hearing aid development (Ahrens et al, 2019; Grimm et al, 2019; Huisman et al, 2019; Pausch & Fels, 2020; Pausch & Fels, 2020; Rindel, 2001; Schutte et al, 2019; Seeber et al, 2010), particularly in the context of ecological validity, where the results of speech-in-noise and hearing test with simplified “synthetic” stimuli have been questioned with regard to real-world outcomes (Cord et al, 2007; Jerger, 2009; Keidser et al, 2020; Miles et al, 2020).…”

Acoustic scene complexity affects motion behavior during speech perception in audio-visual multi-talker virtual environments

“…Wearing a VR headset likely affected the listener's HRTFs 32 , potentially resulting in localization errors and different movement behavior compared to real-world scenarios. However, studies by Ahrens et al 15 and Huisman et al 45 suggest that the influence of VR headset on sound localization accuracy is minimal when ample visual information is provided. Nonetheless, the impact of VR headsets on movement behavior remains uncertain.…”

Cross-modal correspondence enhances elevation localization in visual-to-auditory sensory substitution

“…The participants tended to overestimate the lateral position of the lateral targets with the three visual-to-auditory encodings: a shift to the left for targets on the left, and a shift to the right for targets on the right. Some studies also showed an overestimation pattern of lateral sound sources while using non-individualized HRTFs (Wenzel et al, 1993 ), in a virtual environment while being blindfolded (Ahrens et al, 2019 ), using ambisonics (Huisman et al, 2021 ), and even with real sound sources (Oldfield and Parker, 1984 ; Makous and Middlebrooks, 1990 ). A possibility to decrease this overestimation might be to rescale the used HRTF positions to fit to the perceived ones.…”