Multisensory stimuli improve relative localisation judgments compared to unisensory auditory or visual stimuli

Freeman, Laura; Wood, Katherine C.; Bizley, Jennifer K.

doi:10.1121/1.5042759

Cited by 7 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We assume that the present results can be explained in terms of the lack of retinovisuomotor calibration of the auditory space with blindness and by the specificities of the multisensory mechanisms of spatial calibration that replace visual calibration in the blind (Lewald, 2002a(Lewald, , 2002b(Lewald, , 2013Zwiers et al, 2001aZwiers et al, , 2001bZwiers et al, , 2003. In this context, it is important to note that in normalsighted persons, visual localization acuity is maximal in the foveal region and decreases substantially with increasing eccentricity (Charbonneau et al, 2013;Freeman, Wood, & Bizley, 2018;Mateeff & Gourevich, 1983;Perrott, Costantino, & Cisneros, 1993). As mentioned earlier, auditory localization acuity also increases with increasing eccentricity (Blauert, 1997;Carlile et al, 2016;Charbonneau et al, 2013;Freeman et al, 2018;Makous & Middlebrooks, 1990;Mills, 1958;R€ oder et al, 1999;Wood & Bizley, 2015).…”

Section: Discussionsupporting

confidence: 63%

“…In this context, it is important to note that in normalsighted persons, visual localization acuity is maximal in the foveal region and decreases substantially with increasing eccentricity (Charbonneau et al, 2013;Freeman, Wood, & Bizley, 2018;Mateeff & Gourevich, 1983;Perrott, Costantino, & Cisneros, 1993). As mentioned earlier, auditory localization acuity also increases with increasing eccentricity (Blauert, 1997;Carlile et al, 2016;Charbonneau et al, 2013;Freeman et al, 2018;Makous & Middlebrooks, 1990;Mills, 1958;R€ oder et al, 1999;Wood & Bizley, 2015). Furthermore, the cross-modal bias (i.e., the ability of a visual signal to influence the localization of an auditory target) and the ventriloquism effect (i.e., the visual capture of sounds with presentation of spatially incongruent bimodal targets) have been shown to be maximal at central locations and to decrease with eccentricity (Charbonneau et al, 2013;Hairston et al, 2003;Lewald & Guski, 2003).…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Auditory Space Perception in the Blind: Horizontal Sound Localization in Acoustically Simple and Complex Situations

Feierabend¹,

Karnath

Lewald

2019

Perception

View full text Add to dashboard Cite

Even though it is widely believed that cross-modal neuroplasticity in blindness results in enhanced auditory spatial abilities, a growing number of studies also indicate disadvantages of blind persons for specific, more demanding audiospatial tasks. Here, the effects of blindness on horizontal sound localization were compared for simple localization of target sounds presented in isolation and target localization in an acoustically complex (cocktail-party) situation with multiple distractor sounds. While performances of blind and sighted subjects were similar in the cocktail-party setting, a significant disadvantage of blind subjects was found for simple localization of sound sources presented in the center of the frontal space, with the mean absolute error being more than twice that of controls for targets in the median plane. The latter result could be due to the lack of visual calibration of auditory space, which may, in sighted persons, be most effective in the region of maximum visual acuity. On the other hand, the normal performance of blind persons in cocktail-party localization could be explained by the fact that this task, in addition to genuine spatial analysis, critically involved spectrotemporal analysis of the auditory scene to separate different sources for which superior performance is known from previous research.

show abstract

Section: Discussionsupporting

confidence: 63%

Section: Discussionmentioning

confidence: 99%

Auditory Space Perception in the Blind: Horizontal Sound Localization in Acoustically Simple and Complex Situations

Feierabend¹,

Karnath

Lewald

2019

Perception

View full text Add to dashboard Cite

show abstract

“…In virtual audio-visual scenes the audio is not presented in isolation, but in combination with visual information, which is known to strongly influence sound localization (e.g., [15][16][17]). When audio and visual stimuli are presented in close temporal and spatial proximity, they are integrated into one common percept, increasing the accuracy and precision of localization (e.g., [18][19][20]). As a result of this process, when the audio and visual stimuli are not exactly at the same position, but still integrated, the perceived location of the auditory stimuli is shifted strongly towards that of the visual stimulus (the so called 'ventriloquist effect', e.g., [18,[21][22][23]).…”

Section: Introductionmentioning

confidence: 99%

Sound source localization in virtual reality with ambisonics sound reproduction

Huisman¹,

Ahrens²,

MacDonald³

2021

Preprint

View full text Add to dashboard Cite

To reproduce realistic audio-visual scenarios in the laboratory, ambisonics is often used to reproduce a sound field over loudspeakers and virtual reality (VR) glasses are used to present visual information. Both technologies have been shown to be suitable for research. However, the combination of both technologies, ambisonics and VR glasses, might affect the spatial cues for auditory localization and thus, the localization percept. Here, we investigated how VR glasses affect the localization of virtual sound sources on the horizontal plane produced using either 1st, 3rd, 5th or 11th order ambisonics with and without visual information. Results showed that with 1st order ambisonics the localization error is larger than with the higher orders, while the differences across the higher orders were small.The physical presence of the VR glasses without visual information increased the perceived lateralization of the auditory stimuli by on average about 2°, especially in the right hemisphere. Presenting visual information about the environment and potential sound sources did reduce this HMD-induced shift, however it could not fully compensate for it. While the localization performance itself was affected by the ambisonics order, there was no interaction between the ambisonics order and the effect of the HMD. Thus, the presence of VR glasses can alter acoustic localization when using ambisonics sound reproduction, but visual information can compensate for most of the effects. As such, most use cases for VR will be unaffected by these shifts in the perceived location of auditory stimuli.

show abstract

“…In virtual audio-visual scenes the audio is not presented in isolation, but in combination with visual information, which is known to strongly influence sound localization (e.g., Dufour et al, 2002;Tabry et al, 2013;Gori et al, 2014). When audio and visual stimuli are presented in close temporal and spatial proximity, they are integrated into one common percept, increasing the accuracy and precision of localization (e.g., Alais and Burr, 2004;Odegaard et al, 2015;Freeman et al, 2018). As a result of this process, when the audio and visual stimuli are not exactly at the same position, but still integrated, the perceived location of the auditory stimuli is shifted strongly towards that of the visual stimulus (the so called "ventriloquist effect,", e.g., Alais and Burr, 2004;Jackson, 1953;Lewald and Guski, 2003;Thurlow and Jack, 1973).…”

Section: Introductionmentioning

confidence: 99%

Ambisonics Sound Source Localization With Varying Amount of Visual Information in Virtual Reality

Huisman

Ahrens

MacDonald

2021

Front. Virtual Real.

View full text Add to dashboard Cite

To reproduce realistic audio-visual scenarios in the laboratory, Ambisonics is often used to reproduce a sound field over loudspeakers and virtual reality (VR) glasses are used to present visual information. Both technologies have been shown to be suitable for research. However, the combination of both technologies, Ambisonics and VR glasses, might affect the spatial cues for auditory localization and thus, the localization percept. Here, we investigated how VR glasses affect the localization of virtual sound sources on the horizontal plane produced using either 1st-, 3rd-, 5th- or 11th-order Ambisonics with and without visual information. Results showed that with 1st-order Ambisonics the localization error is larger than with the higher orders, while the differences across the higher orders were small. The physical presence of the VR glasses without visual information increased the perceived lateralization of the auditory stimuli by on average about 2°, especially in the right hemisphere. Presenting visual information about the environment and potential sound sources did reduce this HMD-induced shift, however it could not fully compensate for it. While the localization performance itself was affected by the Ambisonics order, there was no interaction between the Ambisonics order and the effect of the HMD. Thus, the presence of VR glasses can alter acoustic localization when using Ambisonics sound reproduction, but visual information can compensate for most of the effects. As such, most use cases for VR will be unaffected by these shifts in the perceived location of the auditory stimuli.

show abstract

Multisensory stimuli improve relative localisation judgments compared to unisensory auditory or visual stimuli

Cited by 7 publications

References 23 publications

Auditory Space Perception in the Blind: Horizontal Sound Localization in Acoustically Simple and Complex Situations

Auditory Space Perception in the Blind: Horizontal Sound Localization in Acoustically Simple and Complex Situations

Sound source localization in virtual reality with ambisonics sound reproduction

Ambisonics Sound Source Localization With Varying Amount of Visual Information in Virtual Reality

Contact Info

Product

Resources

About