A Representation of Effort in Decision-Making and Motor Control

Abstract

Given two rewarding stimuli, animals tend to choose the more rewarding (or less effortful) option. However, they also move faster toward that stimulus [1-5]. This suggests that reward and effort not only affect decision-making, they also influence motor control [6, 7]. How does the brain compute the effort requirements of a task? Here, we considered data acquired during walking, reaching, flying, or isometric force production. In analyzing the decision-making and motor-control behaviors of various animals, we considered the possibility that the brain may estimate effort objectively, via the metabolic energy consumed to produce the action. We measured the energetic cost of reaching and found that, like walking, it was convex in time, with a global minimum, implying that there existed a movement speed that minimized effort. However, reward made it worthwhile to be energetically inefficient. Using a framework in which utility of an action depended on reward and energetic cost, both discounted in time, we found that it was possible to account for a body of data in which animals were free to choose how to move (reach slow or fast), as well as what to do (walk or fly, produce force F1 or F2). We suggest that some forms of decision-making and motor control may share a common utility in which the brain represents the effort associated with performing an action objectively via its metabolic energy cost and then, like reward, temporally discounts it as a function of movement duration.

Direction-dependent arm kinematics reveal optimal integration of gravity cues

“…Furthermore, minimizing other torque-related cost functions failed to predict direction-dependent kinematics (Figure 1F). The present results therefore strongly support effort minimization in humans, further extending the growing idea that perceived effort plays an important role in the tailoring of human motor as well as non-motor behaviors (Bramble and Lieberman, 2004; Walton et al, 2006; Mazzoni et al, 2007; Carrier et al, 2011; Kurzban et al, 2013; Selinger et al, 2015; Farshchiansadegh et al, 2016; Shadmehr et al, 2016). …”

Single reach plans in dorsal premotor cortex during a two-target task

“…7c, top row). The inconsistency of the planning preferences, both within and across subjects, suggests that they arose as part of conscious task strategies rather than from an ingrained predilection for minimizing exertion or some other physiological cost function 35–37 .

Fig. 7Preferential reach planning in PMd during two-target trials.

…”

What makes a reach movement effortful? Physical effort discounting supports common minimization principles in decision making and motor control

“…To explain such a counterintuitive effect, the authors assume that effort costs are subject to the same temporal discounting as can typically be observed for reward in economic choice behavior [22]. This hypothetical explanation is, however, not applicable to our results, as we kept the total duration of trials constant.…”

“…When considering whole movements and not single muscles, the energy rate is linearly related to the squared speed for walking [29]. Reaching shows similar relationships in which, at constant durations, the energy expended for movements depends almost linearly (exponent = 1.1) on distance [22]. The use of energetic cost as a decision factor is thus neither consistent with the lacking amplitude dependency in our experiment 1 nor with the quadratic force–effort relationship observed in experiment 2.…”