The Effects of Cue Reliability on Crossmodal Recalibration in Adults and Children

Abstract

Reliability-based cue combination is a hallmark of multisensory integration, while the role of cue reliability for crossmodal recalibration is less understood. The present study investigated whether visual cue reliability affects audiovisual recalibration in adults and children. Participants had to localize sounds, which were presented either alone or in combination with a spatially discrepant high- or low-reliability visual stimulus. In a previous study we had shown that the ventriloquist effect (indicating multisensory integration) was overall larger in the children groups and that the shift in sound localization toward the spatially discrepant visual stimulus decreased with visual cue reliability in all groups. The present study replicated the onset of the immediate ventriloquist aftereffect (a shift in unimodal sound localization following a single exposure of a spatially discrepant audiovisual stimulus) at the age of 6–7 years. In adults the immediate ventriloquist aftereffect depended on visual cue reliability, whereas the cumulative ventriloquist aftereffect (reflecting the audiovisual spatial discrepancies over the complete experiment) did not. In 6–7-year-olds the immediate ventriloquist aftereffect was independent of visual cue reliability. The present results are compatible with the idea of immediate and cumulative crossmodal recalibrations being dissociable processes and that the immediate ventriloquist aftereffect is more closely related to genuine multisensory integration.

Audiovisual spatial recalibration but not integration is shaped by early sensory experience

“…A similar result pattern emerged for immediate (trial-by-trial) recalibration: Despite an additional visual VAEc in CC individuals, an additional visual VAEi was not detected, whereas a typical auditory VAEi was revealed. In fact, the size of the auditory VAEi and VE was significantly correlated in the CC group as has been previously demonstrated in normally sighted individuals ( Rohlf et al., 2021 ). Thus, the present results provide further evidence for a dissociation of cumulative crossmodal spatial recalibration (VAEc) and both immediate crossmodal recalibration (VAEi) and multisensory integration (VE), suggesting that the latter two processes originate from overlapping neural mechanisms ( Park and Kayser, 2019 ).…”

“…There is multiple independent evidence suggesting that multisensory integration (VE) and crossmodal recalibration (VAEc) reflect at least partially distinct mechanisms: Event-related potential studies revealed an early versus late modulation for the VAEc and VE, respectively ( Bonath et al., 2007 ; Bruns et al., 2011 ; Bruns and Röder, 2010 ). Prospective studies in children have recently observed that the VE emerges prior to the VAEc ( Rohlf et al., 2020 , 2021 ), thus providing developmental evidence for a dissociation of both effects. Moreover, the VAEc was found to be independent of cue reliability whereas the VE, in accord with an extensive literature ( Alais and Burr, 2019 ), varied with cue reliability even in the youngest (5 years old) children tested ( Rohlf et al., 2020 , 2021 ).…”

“…Prospective studies in children have recently observed that the VE emerges prior to the VAEc ( Rohlf et al., 2020 , 2021 ), thus providing developmental evidence for a dissociation of both effects. Moreover, the VAEc was found to be independent of cue reliability whereas the VE, in accord with an extensive literature ( Alais and Burr, 2019 ), varied with cue reliability even in the youngest (5 years old) children tested ( Rohlf et al., 2020 , 2021 ). Zaidel et al.…”

“…A study in children has recently found that crossmodal spatial recalibration does not arise before middle childhood (with immediate recalibration occurring before cumulative recalibration) and, thus, develops later than the ability to integrate auditory and visual spatial input for localization ( Rohlf et al., 2020 ). Moreover, it has been suggested that the weights assigned to each sensory cue during cumulative recalibration in adulthood do not depend on their precision but rather seem to be fixed ( Rohlf et al., 2021 ; Zaidel et al., 2011 ; but see Hong et al., 2021 ), whereas they are dependent on cue reliability in both immediate recalibration and multisensory integration ( Rohlf et al., 2021 ). Thus, the late developmental onset of cumulative crossmodal recalibration might be a consequence of the need to acquire the relative weighting of individual sensory cues through extensive experience which is consecutively stabilized by an elaboration of anatomical connectivity during development ( Linkenhoker et al., 2005 ).…”

Musical training refines audiovisual integration but does not influence temporal recalibration

“…That is, observers who perceived simultaneity between ashes and tones over a narrower range of stimulus onset asynchronies (SOAs) tended to experience less recalibration, supporting the notion that the brain should only compensate for sensory misalignment when the cues are perceived to belong to the same event. This nding was corroborated by some later studies [9][10][11] and a similar association has also been found in the spatial domain where the precision of integration predicts rapid recalibration [12,13] but not cumulative recalibration [12,14] . However, more recent work has suggested that in the temporal domain, the precision of integration does not predict the magnitude of recalibration.…”