Highlights d Multisensory integration develops prior to unsupervised crossmodal recalibration d Multisensory integration and crossmodal learning are dissociable processes d Crossmodal recalibration in humans emerges first for short timescales d Crossmodal recalibration requiring accumulation of evidence develops last

Reliability-based cue combination is a hallmark of multisensory integration, while the role of cue reliability for crossmodal recalibration is less understood. The present study investigated whether visual cue reliability affects audiovisual recalibration in adults and children. Participants had to localize sounds, which were presented either alone or in combination with a spatially discrepant high- or low-reliability visual stimulus. In a previous study we had shown that the ventriloquist effect (indicating multisensory integration) was overall larger in the children groups and that the shift in sound localization toward the spatially discrepant visual stimulus decreased with visual cue reliability in all groups. The present study replicated the onset of the immediate ventriloquist aftereffect (a shift in unimodal sound localization following a single exposure of a spatially discrepant audiovisual stimulus) at the age of 6–7 years. In adults the immediate ventriloquist aftereffect depended on visual cue reliability, whereas the cumulative ventriloquist aftereffect (reflecting the audiovisual spatial discrepancies over the complete experiment) did not. In 6–7-year-olds the immediate ventriloquist aftereffect was independent of visual cue reliability. The present results are compatible with the idea of immediate and cumulative crossmodal recalibrations being dissociable processes and that the immediate ventriloquist aftereffect is more closely related to genuine multisensory integration.

During development internal models of the sensory world must be acquired which have to be continuously adapted later. We used event-related potentials (ERP) to test the hypothesis that infants extract crossmodal statistics implicitly while adults learn them when task relevant. Participants were passively exposed to frequent standard audio-visual combinations (A1V1, A2V2, p=0.35 each), rare recombinations of these standard stimuli (A1V2, A2V1, p=0.10 each), and a rare audio-visual deviant with infrequent auditory and visual elements (A3V3, p=0.10). While both six-month-old infants and adults differentiated between rare deviants and standards involving early neural processing stages only infants were sensitive to crossmodal statistics as indicated by a late ERP difference between standard and recombined stimuli. A second experiment revealed that adults differentiated recombined and standard combinations when crossmodal combinations were task relevant. These results demonstrate a heightened sensitivity for crossmodal statistics in infants and a change in learning mode from infancy to adulthood.

22While adults have to continuously adapt their internal representations of the sensory world, infants 23 need to first acquire these models. We used event-related potentials to test the hypothesis that infants 24 extract crossmodal statistics implicitly while adults learn them when task relevant. Six-month-old 25 infants and adults were passively exposed to frequent standard audio-visual combinations (A1V1, 26 A2V2, p=0.35 each), rare recombinations of the standard stimuli (A1V2, A2V1, p=0.10 each), and a rare 27 deviant audio-visual combination with an infrequent auditory and visual element (A3V3, p=0.10). 28While both infants and adults differentiated between rare deviants and standards at early processing 29 stages, only infants discriminated standards from recombined stimuli at a later processing stage. A 30 second experiment revealed that adults discriminated recombined from standard combinations only 31 when crossmodal combinations were task relevant. These results demonstrate a heightened sensitivity 32 for crossmodal statistics in infants and a change in learning mode from infancy to adulthood.

Event-related potentials (ERP) to the second stimulus of a pair are known to be reduced in amplitude. The magnitude of this ‘refractoriness’ is modulated by both the interstimulus interval and the similarity between the two stimuli. Intramodal refractoriness is interpreted as an index of a temporary decrement in neural responsiveness. So, cross-modal refractoriness might be an indicator of shared neural generators between modalities. We analysed oscillatory neuronal activity while participants were engaged in an oddball paradigm with auditory (4000 Hz, 50 ms-long, 90 db, bilateral) and tactile stimuli (50 ms-long, 125 Hz-vibrations, index fingers) presented in a random order with an ISI of either 1000 or 2000 ms. Participants were required to detect rare tactile (middle fingers) and auditory deviants (600 Hz). A time–frequency analysis of the brain response to the second stimulus of each pair (T-T, A-A, T-A and A-T) contrasting Short and Long ISIs revealed a reduced refractory effect after Long ISI with respect to Short ISI, in all pairs (both intramodal and cross-modal). This emerged as a broadly distributed increase of evoked theta activity (3–7 Hz, 100–500 ms). Only in intramodal tactile pairs and cross-modal tactile-auditory pairs we also observed that Long ISI with respect to Short ISI determined a decrease of induced alpha (8–12 Hz, 200–700 ms), a typical sign of enhanced neural excitability and thus decreased refractoriness. These data suggest that somatosensory and auditory cortices display different neural markers of refractoriness and that the auditory cortex might have a stronger low level degree of influence on the tactile cortex than vice-versa.