Motor Adaptation as a Process of Reoptimization

Abstract

Adaptation is sometimes viewed as a process in which the nervous system learns to predict and cancel effects of a novel environment, returning movements to near baseline (unperturbed) conditions. An alternate view is that cancellation is not the goal of adaptation. Rather, the goal is to maximize performance in that environment. If performance criteria are well defined, theory allows one to predict the reoptimized trajectory. For example, if velocity-dependent forces perturb the hand perpendicular to the direction of a reaching movement, the best reach plan is not a straight line but a curved path that appears to overcompensate for the forces. If this environment is stochastic (changing from trial to trial), the reoptimized plan should take into account this uncertainty, removing the overcompensation. If the stochastic environment is zero-mean, peak velocities should increase to allow for more time to approach the target. Finally, if one is reaching through a via-point, the optimum plan in a zero-mean deterministic environment is a smooth movement but in a zero-mean stochastic environment is a segmented movement. We observed all of these tendencies in how people adapt to novel environments. Therefore, motor control in a novel environment is not a process of perturbation cancellation. Rather, the process resembles reoptimization: through practice in the novel environment, we learn internal models that predict sensory consequences of motor commands. Through reward-based optimization, we use the internal model to search for a better movement plan to minimize implicit motor costs and maximize rewards.

CNS Learns Stable, Accurate, and Efficient Movements Using a Simple Algorithm

“…Current neurophysiological models able to predict trial to trial modifications of force or torque (Kawato et al, 1987;Katayama and Kawato, 1993;Gribble and Ostry, 2000;Thoroughman and Shadmehr, 2000;Donchin et al, 2003;Emken et al, 2007) and corresponding nonlinear adaptive controllers for robots (Slotine and Li, 1991;, which use a monotonic antisymmetric (in most cases, linear) update of the feedforward command, have no explicit mechanism to alter the limb impedance independently from joint torque (or limb posture), and, therefore, cannot learn to compensate for unstable dynamics (Osu et al, 2003). Models based exclusively on optimization of cost functions such as minimization of end-point variance and/or muscle activation (Burdet and Milner, 1998;Harris and Wolpert, 1998;Stroeve, 1999;Todorov, 2000;Todorov and Jordan, 2002;Guigon et al, 2007;Trainin et al, 2007;Izawa et al, 2008) can only predict final learning outcomes, whereas our model can account for the complete progression of experimentally observed changes in force and impedance throughout learning. This algorithm, when combined with a method for generalization (Donchin et al, 2003), and a method for storing and accessing multiple internal representations (Haruno et al, 2001) could provide a powerful description of motor adaptation.…”

“…Our novel algorithm learns the time-varying motor commands to individual muscles that produce the same force and mechanical impedance observed when humans adapt to changes in environmental forces, including those arising from instability in the environment. It departs significantly from algorithms based on optimization (Burdet and Milner, 1998;Harris and Wolpert, 1998;Stroeve, 1999;Todorov, 2000;Todorov and Jordan, 2002;Guigon et al, 2007;Trainin et al, 2007;Izawa et al, 2008) as it predicts the transients of learning, as well as from existing supervised learning schemes (Kawato et al, 1987;Slotine and Li, 1991;Katayama and Kawato, 1993;Gribble and Ostry, 2000;Thoroughman and Shadmehr, 2000;Donchin et al, 2003;Emken et al, 2007) because they have no mechanism to counteract mechanical instability.…”

Use-Dependent and Error-Based Learning of Motor Behaviors

“…First, the solution for the redundant task may have been chosen, because it constitutes the optimal solution under the task requirements (Izawa et al, 2008). Although we cannot exclude the possibility that successful task performance can partly account for the high use-dependent learning rate found in experiment 3 compared with experiments 1 and 2 (see Discussion), we believe that it is unlikely that an optimization process alone can account for the observed adaptation solution.…”

Cerebellar motor learning: are environment dynamics more important than error size?

“…This means that control subjects overcom- pensate for the field early in movement when the curl field forces are small. This has been suggested to be the optimal strategy to minimize effort during adaptation to a velocitydependent curl field (Izawa et al 2008). Similar to control subjects, patients also overcompensate for the CW field as early as 150 ms.…”