Motor Learning with Augmented Feedback: Modality-Dependent Behavioral and Neural Consequences

Abstract

Sensory information is critical to correct performance errors online during the execution of complex tasks and can be complemented by augmented feedback (FB). Here, 2 groups of participants acquired a new bimanual coordination pattern under different augmented FB conditions: 1) visual input reflecting coordination between the 2 hands and 2) auditory pacing integrating the timing of both hands into a single temporal structure. Behavioral findings revealed that the visual group became dependent on this augmented FB for performance, whereas the auditory group performed equally well with or without augmented FB by the end of practice. Functional magnetic resonance imaging (fMRI) results corroborated these behavioral findings: the visual group showed neural activity increases in sensory-specific areas during practice, supporting increased reliance on augmented FB. Conversely, the auditory group showed a neural activity decrease, specifically in areas associated with cognitive/sensory monitoring of motor task performance, supporting the development of a control mode that was less reliant on augmented FB sources. Finally, some remnants of brain activity in sensory-specific areas in the absence of augmented FB were found for the visual group only, illustrating ongoing reliance on these areas. These findings provide the first neural account for the "guidance hypothesis of information FB," extensively supported by behavioral research.

Sonification as Concurrent Augmented Feedback for Motor Skill Learning and the Importance of Mapping Design

“…Clearly, this kind of concurrent visual feedback is at its most effective when the learner is deprived of some awareness of the fundamental kinematics of the motor task in question. Kovacs et al [35] did not test retention without feedback, but one can be fairly confident based on related studies [27,39] that a pronounced guidance effect would be evident. In the case of Lissajous-style feedback, what was originally intended as a guide to help learning an action has swallowed the task to be learned, replacing it completely.…”

“…The source of feedback upon which a learner had previously relied is no longer available to guide performance, and the learner must reweight to a comparatively unused source. The result is impaired performance-monitoring ability [27]. The ability to detect signals corresponding to both good and bad performance using internal monitoring mechanisms (i.e.…”

“…This dormancy is especially true for the kinds of laboratory-friendly motor tasks employed in experimental research into concurrent feedback, e.g. isometric force production [24], bimanual coordination [27] and lever manipulation [25]. Kinetic impacts and interactions with the environment are the normal means by which the human body produces sound from motor activity, as the frequency of movements themselves are below the level of human hearing [31].…”

The Effects of Motor Imagery After a Variety of Motor Learning Times on Excitability of Spinal Motor Neurons and Accurate Motion

“…Conversely, the correction time should be decreased if there is no improvement in the accuracy of motion. Ronsse et al [23] reported that the effectiveness of motor learning with the use of visual feedback was decreased over time with periodic flexion and extension at both wrist joints. Ohashi et al [24] reported that the information pertaining to the intensity of force in an isometric contraction task was decreased over the course of time.…”

A Survey of Multimodal Systems and Techniques for Motor Learning

“…Wallis et al [60] implemented real-time motion sonification using music to impart movement in stroke rehabilitation tasks. Ronsse et al [61] demonstrated that for the pacing and temporal coordination of hand movements during complex bimanual tasks, real-time audio feedback can outperform visual feedback in terms of skill retention. When asked to complete the task without feedback, participants who had received visual feedback retained less of the coordination learned in training than those with audio feedback.…”