Reaching to sounds in virtual reality: A multisensory-motor approach to promote adaptation to altered auditory cues

Valzolgher, Chiara; Verdelet, Grégoire; Salemme, Roméo; Lombardi, Luigi; Gaveau, Valérie; Farnè, Alessandro; Pavani, Francesco

doi:10.1016/j.neuropsychologia.2020.107665

Cited by 22 publications

(59 citation statements)

References 59 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The present study examined sound localization abilities of young BCI users (aged 8–17 years old) compared to NH peers in reaching space. This is a portion of space in which many social interactions occur in daily life, and in which we interact with sound sources through avoidance and reaching movements (e.g., Valzolgher et al, 2020a ). To this aim we used a portable virtual reality and kinematic tracking system which allowed us to deliver real sounds within reaching distance from the listener, at reproducible head-centered coordinates.…”

Section: Discussionmentioning

confidence: 99%

“…Our new approach for measuring spatial hearing in the near-field has been validated in a virtual reality platform easy to access in clinical practice (HTC VIVE; Verdelet et al 2019 ; Valzolgher et al 2020a , b ). These first results give new information about children’s performance, both when they have typical hearing, and when they are deafness but fitted with BCI.…”

Section: Discussionmentioning

confidence: 99%

“…This portion of space is particularly relevant for social interactions, where fast motor responses are needed in case of an approaching auditory object (e.g., a bee), when reaching toward a sound source (e.g., a musical toy) or when orienting towards a nearby talker ( Kolarik et al 2016 ). Finally, a recent study ( Valzolgher et al 2020a ) has shown that the possibility to interact with near-field sounds by reaching to the sound sources directly can promote head movements during listening in case of simulated hearing impairment (monaural ear-plugging) and result in faster adaptation altered auditory cues.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Spatial Hearing Difficulties in Reaching Space in Bilateral Cochlear Implant Children Improve With Head Movements

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Spatial Hearing Difficulties in Reaching Space in Bilateral Cochlear Implant Children Improve With Head Movements

et al. 2021

Self Cite

View full text Add to dashboard Cite

show abstract

“…In this recent work [ 25 ] we also documented that reaching to sounds can ameliorate sound localization with one ear plugged on a trial-by-trial basis. However, it remains unclear if participants improved because of practice with the specific auditory task or if they learned new and effective ways to adapt to the altered auditory cues instead.…”

Section: Introductionmentioning

confidence: 87%

“…In a recent study [ 25 ], the above-mentioned issue was addressed by asking to one-ear-plugged normal hearing participants to identify the position of sounds by reaching or by naming the labels associated with each speaker. The manipulation was performed across groups, using virtual reality (VR).…”

Section: Introductionmentioning

confidence: 99%

Adapting to altered auditory cues: Generalization from manual reaching to head pointing

et al. 2022

Self Cite

View full text Add to dashboard Cite

Localising sounds means having the ability to process auditory cues deriving from the interplay among sound waves, the head and the ears. When auditory cues change because of temporary or permanent hearing loss, sound localization becomes difficult and uncertain. The brain can adapt to altered auditory cues throughout life and multisensory training can promote the relearning of spatial hearing skills. Here, we study the training potentials of sound-oriented motor behaviour to test if a training based on manual actions toward sounds can learning effects that generalize to different auditory spatial tasks. We assessed spatial hearing relearning in normal hearing adults with a plugged ear by using visual virtual reality and body motion tracking. Participants performed two auditory tasks that entail explicit and implicit processing of sound position (head-pointing sound localization and audio-visual attention cueing, respectively), before and after having received a spatial training session in which they identified sound position by reaching to auditory sources nearby. Using a crossover design, the effects of the above-mentioned spatial training were compared to a control condition involving the same physical stimuli, but different task demands (i.e., a non-spatial discrimination of amplitude modulations in the sound). According to our findings, spatial hearing in one-ear plugged participants improved more after reaching to sound trainings rather than in the control condition. Training by reaching also modified head-movement behaviour during listening. Crucially, the improvements observed during training generalize also to a different sound localization task, possibly as a consequence of acquired and novel head-movement strategies.

show abstract

Benefits of active listening during 3D sound localization

et al. 2022

View full text Add to dashboard Cite

In everyday life, sound localization entails more than just the extraction and processing of auditory cues. When determining sound position in three dimensions, the brain also considers the available visual information (e.g., visual cues to sound position) and resolves perceptual ambiguities through active listening behavior (e.g., spontaneous head movements while listening). Here, we examined to what extent spontaneous head movements improve sound localization in 3D—azimuth, elevation, and depth—by comparing static vs. active listening postures. To this aim, we developed a novel approach to sound localization based on sounds delivered in the environment, brought into alignment thanks to a VR system. Our system proved effective for the delivery of sounds at predetermined and repeatable positions in 3D space, without imposing a physically constrained posture, and with minimal training. In addition, it allowed measuring participant behavior (hand, head and eye position) in real time. We report that active listening improved 3D sound localization, primarily by ameliorating accuracy and variability of responses in azimuth and elevation. The more participants made spontaneous head movements, the better was their 3D sound localization performance. Thus, we provide proof of concept of a novel approach to the study of spatial hearing, with potentials for clinical and industrial applications.

show abstract

Reaching to sounds in virtual reality: A multisensory-motor approach to promote adaptation to altered auditory cues

Cited by 22 publications

References 59 publications

Spatial Hearing Difficulties in Reaching Space in Bilateral Cochlear Implant Children Improve With Head Movements

Spatial Hearing Difficulties in Reaching Space in Bilateral Cochlear Implant Children Improve With Head Movements

Adapting to altered auditory cues: Generalization from manual reaching to head pointing

Benefits of active listening during 3D sound localization

Contact Info

Product

Resources

About