Long-term music training tunes how the brain temporally binds signals from multiple senses

Abstract

Practicing a musical instrument is a rich multisensory experience involving the integration of visual, auditory, and tactile inputs with motor responses. This combined psychophysics-fMRI study used the musician's brain to investigate how sensory-motor experience molds temporal binding of auditory and visual signals. Behaviorally, musicians exhibited a narrower temporal integration window than nonmusicians for music but not for speech. At the neural level, musicians showed increased audiovisual asynchrony responses and effective connectivity selectively for music in a superior temporal sulcus-premotor-cerebellar circuitry. Critically, the premotor asynchrony effects predicted musicians' perceptual sensitivity to audiovisual asynchrony. Our results suggest that piano practicing fine tunes an internal forward model mapping from action plans of piano playing onto visible finger movements and sounds. This internal forward model furnishes more precise estimates of the relative audiovisual timings and hence, stronger prediction error signals specifically for asynchronous music in a premotor-cerebellar circuitry. Our findings show intimate links between action production and audiovisual temporal binding in perception.audiovisual synchrony | multisensory integration | sensorimotor learning | crossmodal integration | experience-dependent plasticity P racticing a musical instrument is a rich multisensory experience involving the integration of visual, auditory, and tactile inputs with motor responses. The musician's brain, thus, provides an ideal model to study experience-dependent plasticity in humans (1, 2).Previous research in musicians has focused on neural plasticity affecting unisensory and motor processing. Little is known about how musical expertise alters the integration of inputs from multiple senses. Because musical performance requires precise timing, musical expertise may specifically modulate the temporal binding of sensory signals. Given the variability in physical and neural transmission times, sensory signals do not have to be precisely synchronous but must co-occur within a temporal window that flexibly adapts to the temporal statistics of the sensory inputs as a consequence of music (3) or audiovisual training (4). At the neural level, audiovisual (a)synchrony processing relies on a widespread neural system encompassing subcortical, primary sensory, higher-order association, cerebellar, and premotor areas (5-8).This study used the musician's brain as a model to investigate how long-term sensory-motor experience (i.e., piano practicing) shapes the neural processes underlying temporal binding of auditory and visual signals. We presented subjects with synchronous and asynchronous speech and piano music as two stimulus classes that are both characterized by a rich hierarchical temporal structure but linked to different motor effectors (mouth vs. hand). Comparing the effect of musical expertise on synchrony perception of speech and music allowed us to dissociate generic and context-specific neural mechanisms ...

The unity assumption facilitates cross-modal binding of musical, non-speech stimuli: The role of spectral and amplitude envelope cues

“…This may be explained by musicians' more narrow temporal windows for audiovisual integration in general , and for musical stimuli, more specifically (Lee & Noppeney, 2011), or from differences in motivation between musicians and non-musicians (McAuley, Henry, & Tuft, 2011). Importantly, the effect of match condition did not differ as a result of musical training.…”

“…Accuracy is superior for familiar stimuli (Vatakis & Spence, 2006a), and musical training increases sensitivity to audiovisual asynchrony (Lee & Noppeney 2011). Past findings also suggest that musicians, through their long-term training, acquire internal forward action models that allow them to better predict the auditory outcomes of visual actions (Petrini, Russell & Pollick, 2009).…”

“…For instance, temporal ventriloquism is a process by which a cross-modal temporal conflict (asynchrony) is resolved through cross-modal attraction in time Morein-Zamfir, Soto-Faraco, & Kingston, 2003). The temporal window in which we are tolerant to cross-modal asynchrony is malleable-it can be widened by bottom up factors such as repeated exposure to asynchronous stimuli (Navarra et al, 2005), and shortened through experience such as long-term musical training (Lee & Noppeney, 2011) or short-term perceptual learning (Powers, Hillock & Wallace, 2009). Here we explore how this temporal window can also be influenced by the unity assumption.…”

Rhythm in joint action: psychological and neurophysiological mechanisms for real-time interpersonal coordination

“…Common coding ensures that, when skilled pianists hear sounds [86,87] or see movements [88] associated with a trained piece, they represent these effects in terms of the neural resources necessary to produce them. These representations carry information about the action's temporal dynamics [89], which permits the observer to anticipate the action internally [90]. On this view, action simulation ( §2a) involves accessing this temporal information by activating an action representation.…”

The Plasticity of Brain Networks as a Basis for a Science of Nervous System Rehabilitation

“…We know that there exist Figure 9. [19][20][21] The examination of the differences in the brains of musicians and non-musicians provides an ideal model to compare and examine functional and structural brain plasticity as musicians continuously practice complex motor, auditory, and multimodal skills. We also know that music training in children results in long-term enhancement of visual-spatial, verbal, and mathematical performance.…”