Re-weighting of Sound Localization Cues by Audiovisual Training

Abstract

Sound localization requires the integration in the brain of auditory spatial cues generated by interactions with the external ears, head and body. Perceptual learning studies have shown that the relative weighting of these cues can change in a context-dependent fashion if their relative reliability is altered. One factor that may influence this process is vision, which tends to dominate localization judgments when both modalities are present and induces a recalibration of auditory space if they become misaligned. It is not known, however, whether vision can alter the weighting of individual auditory localization cues. Using virtual acoustic space stimuli, we measured changes in subjects’ sound localization biases and binaural localization cue weights after ∼50 min of training on audiovisual tasks in which visual stimuli were either informative or not about the location of broadband sounds. Four different spatial configurations were used in which we varied the relative reliability of the binaural cues: interaural time differences (ITDs) and frequency-dependent interaural level differences (ILDs). In most subjects and experiments, ILDs were weighted more highly than ITDs before training. When visual cues were spatially uninformative, some subjects showed a reduction in auditory localization bias and the relative weighting of ILDs increased after training with congruent binaural cues. ILDs were also upweighted if they were paired with spatially-congruent visual cues, and the largest group-level improvements in sound localization accuracy occurred when both binaural cues were matched to visual stimuli. These data suggest that binaural cue reweighting reflects baseline differences in the relative weights of ILDs and ITDs, but is also shaped by the availability of congruent visual stimuli. Training subjects with consistently misaligned binaural and visual cues produced the ventriloquism aftereffect, i.e., a corresponding shift in auditory localization bias, without affecting the inter-subject variability in sound localization judgments or their binaural cue weights. Our results show that the relative weighting of different auditory localization cues can be changed by training in ways that depend on their reliability as well as the availability of visual spatial information, with the largest improvements in sound localization likely to result from training with fully congruent audiovisual information.

Microsecond interaural time difference discrimination restored by cochlear implants after neonatal deafness

“…It is well known that the normal auditory system not only combines ITD information with ILD and monaural spectral cues to localize sounds in space, it also adapts strongly to changes in these cues, and can re-weight them depending on their reliability ( Keating et al, 2013b ; Keating et al, 2015 ; Tillein et al, 2016 ; Kumpik et al, 2019 ). Similarly, Jones et al, 2011 demonstrated changes in ITD and ILD thresholds as head size and pinnae grow for up to 6 weeks postnatally in chinchillas.…”

Reweighting of Binaural Localization Cues Induced by Lateralization Training

“…This is in line with Rakerd and Hartmann's (2010) observation that responses follow ILDs in reverberant environments. Furthermore, Kumpik et al (2019) found an even stronger increase in ILD weighting for a condition in which no cue was randomized (i.e., spatially consistent ITDs and ILDs were presented), making it difficult to draw strong conclusions. Considering the changing relative contributions of ITD and ILD depending on stimulus and contextual factors as discussed above, binaural-cue reweighting likely plays a role in adapting to variable acoustic environments.…”

“…Potential reasons for their null result are that (1) the training regimen (i.e., left/ right discrimination) was not sufficiently intuitive, (2) the stimuli were noise bands centered at 500 Hz and thus in a frequency range where only ITDs but not ILDs arise naturally (except for sound sources near the head), (3) the binaural cues used were not sufficiently salient as they were close to the binaural-cue threshold, and (4) auditory and visual stimuli were not presented simultaneously, preventing bottom-up multisensory integration that may be required to induce reweighting. Kumpik et al (2019) presented auditory stimuli with either randomized ITDs (and stable ILDs) or randomized ILDs (and stable ITDs) while participants completed a visual oddball task. They observed an increase in ILD weighting after ITDs were randomized, but no increase in ITD weighting after ILDs were randomized.…”

Binaural Modelling and Spatial Auditory Cue Analysis of 3D-Printed Ears

“…i.e., binaural hearing, humans and animals are able to localize a sound source not only in a 2D space, but also in a three-dimensional (3D) space by analyzing different auditory cues. The brain deciphers the audio cues to predict the direction and distance of the sound source [ 3 ]. The job of the ears is to capture and send natural acoustic signals to the brain for processing.…”