Human motor behavior is constantly adapted through the process of error-based learning. When the motor system encounters an error, its estimate about the body and environment will change, and the next movement will be immediately modified to counteract the underlying perturbation. Here, we show that a second mechanism, use-dependent learning, simultaneously changes movements to become more similar to the last movement. In three experiments, participants made reaching movements toward a horizontally elongated target, such that errors in the initial movement direction did not have to be corrected. Along this task-redundant dimension, we were able to induce use-dependent learning by passively guiding movements in a direction angled by 8°from the previous direction. In a second study, we show that error-based and use-dependent learning can change motor behavior simultaneously in opposing directions by physically constraining the direction of active movements. After removal of the constraint, participants briefly exhibit an error-based aftereffect against the direction of the constraint, followed by a longer-lasting use-dependent aftereffect in the direction of the constraint. In the third experiment, we show that these two learning mechanisms together determine the solution the motor system adopts when learning a motor task.

Humans routinely formulate plans in domains so complex that even the most powerful computers are taxed. To do so, they seem to avail themselves of many strategies and heuristics that efficiently simplify, approximate, and hierarchically decompose hard tasks into simpler subtasks. Theoretical and cognitive research has revealed several such strategies; however, little is known about their establishment, interaction, and efficiency. Here, we use modelbased behavioral analysis to provide a detailed examination of the performance of human subjects in a moderately deep planning task. We find that subjects exploit the structure of the domain to establish subgoals in a way that achieves a nearly maximal reduction in the cost of computing values of choices, but then combine partial searches with greedy local steps to solve subtasks, and maladaptively prune the decision trees of subtasks in a reflexive manner upon encountering salient losses. Subjects come idiosyncratically to favor particular sequences of actions to achieve subgoals, creating novel complex actions or "options."planning | hierarchical reinforcement learning | memoization | pruning H umans and other animals often face complex tasks and environments in which they have to plan and execute long sequences of appropriate actions to achieve distant goals. One can represent the space of future actions and outcomes as a tree; such trees grow inordinately (often exponentially) large as a function of the length of the sequence (i.e., the depth of the tree). Rather little is definitively known about how this computational complexity is addressed. Work in the fields of reinforcement learning and artificial intelligence has suggested a number of heuristics that we describe below, namely, hacking, hierarchies, hoarding, and habitization (1-4). Various tasks have been designed to highlight individual heuristics; though how subjects generate and combine them without clear instruction has not been well characterized (however, see refs. 5 and 6).We previously designed a moderately deep planning problem to elicit a specific heuristic, in this case hacking or pruning of the decision tree (4). However, the task contains many of the elements that make choosing appropriately tricky in general. Thus, we closely examined the nature of, and individual differences between, the performance of subjects, shedding light on the interaction of heuristics in the self-generation of adaptive control when faced with a complex planning problem.Subjects had to plan a path through a maze so as to maximize their cumulative earnings. On each trial, they were placed in a random state and were asked to plan to a depth of 3, 4, or 5 moves ( Fig. 1 A and B). Because each depth involved a binary choice, planning to depths 3, 4, and 5 corresponded to choosing among a set of 8, 16, or 32 possible sequences. We previously found that the large immediate losses at particular branch points in the tree (the red transitions) encouraged subjects to eliminate possibly lucrative subbranches beneath those points ...

Studies with patients and functional magnetic resonance imaging investigations have demonstrated that the cerebellum plays an essential role in adaptation to visuomotor rotation and force field perturbation. To identify cerebellar structures involved in the two tasks, we studied 19 patients with focal lesions after cerebellar infarction. Focal lesions were manually traced on magnetic resonance images and normalized using a new spatially unbiased template of the cerebellum. In addition, we reanalyzed data from 14 patients with cerebellar degeneration using voxel-based morphometry. We found that adjacent regions with only little overlap in the anterior arm area (lobules IV to VI) are important for adaptation in both tasks. Although adaptation to the force field task lay more anteriorly (lobules IV and V), lobule VI was more important for the visuomotor task. In addition, regions in the posterolateral cerebellum (Crus I and II) contributed to both tasks. No consistent involvement of the posterior arm region (lobule VIII) was found. Independence of the two kinds of adaptation is further supported by findings that performance in one task did not correlate to performance in the other task. Our results show that the anterior arm area of the cerebellum is functionally divided into a more posterior part of lobule VI, extending into lobule V, related to visuomotor adaption, and a more anterior part including lobules IV and V, related to force field adaption. The posterolateral cerebellum may process common aspects of both tasks.