Observation of sonified movements engages a basal ganglia frontocortical network

Abstract

BackgroundProducing sounds by a musical instrument can lead to audiomotor coupling, i.e. the joint activation of the auditory and motor system, even when only one modality is probed. The sonification of otherwise mute movements by sounds based on kinematic parameters of the movement has been shown to improve motor performance and perception of movements.ResultsHere we demonstrate in a group of healthy young non-athletes that congruently (sounds match visual movement kinematics) vs. incongruently (no match) sonified breaststroke movements of a human avatar lead to better perceptual judgement of small differences in movement velocity. Moreover, functional magnetic resonance imaging revealed enhanced activity in superior and medial posterior temporal regions including the superior temporal sulcus, known as an important multisensory integration site, as well as the insula bilaterally and the precentral gyrus on the right side. Functional connectivity analysis revealed pronounced connectivity of the STS with the basal ganglia and thalamus as well as frontal motor regions for the congruent stimuli. This was not seen to the same extent for the incongruent stimuli.ConclusionsWe conclude that sonification of movements amplifies the activity of the human action observation system including subcortical structures of the motor loop. Sonification may thus be an important method to enhance training and therapy effects in sports science and neurological rehabilitation.

Joint Action: Mental Representations, Shared Information and General Mechanisms for Coordinating with Others

“…For instance, whilst the mirror system is mostly understood as a visual system sensitive to biological motion information, it is actually also tuned to auditory (Kohler et al, 2002; Bidet-Caulet et al, 2005) and audiovisual information (Lahav et al, 2007). Neuroimaging evidence shows enhanced activation of most parts of the action observation system (medial and superior temporal sulcus, inferior parietal cortex, premotor regions, and subcortical structures) when observing agents’ convergent compared to divergent audiovisual movement patterns (Schmitz et al, 2013). …”

Auditory Coding of Human Movement Kinematics

“…Participants listened to the environmental sound of footsteps presented binaurally and were able to indicate when and to which direction (left or right of auditory field) the sound changed from one side to the other (Bidet-Caulet et al, 2005). Following their fMRI data, Schmitz et al (2013) suppose that specific brain areas (STS, mirror neuron system), which are involved in visual motion perception are activated also during auditory perception of artificial movement sounds as a part of multisensory integration (Lahav et ai, 2007). Another line of argumentation refers to a common code principle when explaining the advantages of multisensory stimuli compared to unisensory stimuli.…”

“…Meanwhile research on biological motion perception has been expanded to other modalities, as to the auditory domain on natural motion sounds (Bidet-Caulet et al, 2005). Current research is indicating that also artificial movement acoustics is feasible to address mechanisms of biological motion perception enabling participants to assess qualitative and quantitative features of gross motor movements Schmitz et al, 2013). But it still remains unclear if complex movement patterns can be identified based on artificial kinematic movement sonification alone and without respective auditory experience.…”

The effect of real-time auditory feedback on learning new characters

“…The relatively close neural connections between auditory and motor areas may explain these audio-motor interactions (Bengtsson et al, 2009;Haueisen & Knöshe, 2001;Schmitz et al, 2013). Furthermore, a growing number of studies report that the audiovisual perception of sonified movement modulates the activity of the multisensory brain areas which then might lead to a more accurate representation of the movement (Bangert et al, 2006;Butler, James, & James, 2011;Le Bel, Pineda, & Sharma, 2009;Scheef et al, 2009;Schmitz et al, 2013). Finally, in addition to their informative characteristics, sounds can be playful and motivate learners (Schaffert, Barrass, & Effenberg, 2009).…”