Motor Adaptation as a Greedy Optimization of Error and Effort

Abstract

Motor adaptation to a novel dynamic environment is primarily thought of as a process in which the nervous system learns to anticipate the environmental forces to eliminate kinematic error. Here we show that motor adaptation can more generally be modeled as a process in which the motor system greedily minimizes a cost function that is the weighted sum of kinematic error and effort. The learning dynamics predicted by this minimization process are a linear, auto-regressive equation with only one state, which has been identified previously as providing a good fit to data from force-field-type experiments. Thus we provide a new theoretical result that shows how these previously identified learning dynamics can be viewed as arising from an optimization of error and effort. We also show that the coefficients of the learning dynamics must fall within a specific range for the optimization model to be valid and verify with experimental data from walking in a force field that they indeed fall in this range. Finally, we attempted to falsify the model by performing experiments in two conditions (repeated exposure to a force field, exposure to force fields of different strengths) for which the single-state, auto-regressive equation might be expected to not fit the data well. We found however that the equation adequately captured the pattern of errors and thus conclude that motor adaptation to a force field can be approximated as an optimization of effort and error for a range of experimental conditions.

CNS Learns Stable, Accurate, and Efficient Movements Using a Simple Algorithm

“…Motor learning uses sensory feedback to modify feedforward commands and improve performance (Johansson and Cole, 1994). Existing learning schemes from neuroscience or robotics based on iterative learning or adaptive control change the feedforward command based on a monotonic function of the kinematic error in joint or muscle space (Kawato et al, 1987;Slotine and Li, 1991;Katayama and Kawato, 1993;Gribble and Ostry, 2000;Thoroughman and Shadmehr, 2000;Donchin et al, 2003;Emken et al, 2007) (see Fig. 1 A).…”

“…Our novel algorithm learns the time-varying motor commands to individual muscles that produce the same force and mechanical impedance observed when humans adapt to changes in environmental forces, including those arising from instability in the environment. It departs significantly from algorithms based on optimization (Burdet and Milner, 1998;Harris and Wolpert, 1998;Stroeve, 1999;Todorov, 2000;Todorov and Jordan, 2002;Guigon et al, 2007;Trainin et al, 2007;Izawa et al, 2008) as it predicts the transients of learning, as well as from existing supervised learning schemes (Kawato et al, 1987;Slotine and Li, 1991;Katayama and Kawato, 1993;Gribble and Ostry, 2000;Thoroughman and Shadmehr, 2000;Donchin et al, 2003;Emken et al, 2007) because they have no mechanism to counteract mechanical instability.…”

Development of novel devices for upper-extremity rehabilitation

“…This result is in agreement with studies that found that minimization of control effort plays a role in kinetic adaptation to novel dynamic environments, but is less effective than minimization of execution errors [182,51].…”

“…Still, the disadaptation occurred at a much slower rate than when kinematical errors were allowed to occur. Further evidence for a contribution of muscular effort in adaptation was recently provided by Emken and colleagues [51]. They examined the adaptation to an externally applied force field during the swing phase of walking and showed that a model describing the temporal evolution of error [208,183] could be derived from minimization of a cost function that is a weighted sum of the execution error and control effort.…”

“…Also computational theories of motor learning exist in which both control and execution effort drives motor learning [205]. Experimental shows that minimization of both control effort and execution error characterizes learning dynamics [183,51]. In these studies similar haptic devices as being used in neurorehabilitation were applied to study how people adapt when exposed to force fields applied to moving limbs while exposed to a novel dynamical environment.…”