How people initiate energy optimization and converge on their optimal gaits

Abstract

A central principle in motor control is that the coordination strategies learned by our nervous system are often optimal. Here, we combined human experiments with computational reinforcement learning models to study how the nervous system navigates possible movements to arrive at an optimal coordination. Our experiments used robotic exoskeletons to reshape the relationship between how participants walk and how much energy they consume. We found that while some participants used their relatively high natural gait variability to explore the new energetic landscape and spontaneously initiate energy optimization, most participants preferred to exploit their originally preferred, but now suboptimal, gait. We could nevertheless reliably initiate optimization in these exploiters by providing them with the experience of lower cost gaits, suggesting that the nervous system benefits from cues about the relevant dimensions along which to reoptimize its coordination. Once optimization was initiated, we found that the nervous system employed a local search process to converge on the new optimum gait over tens of seconds. Once optimization was completed, the nervous system learned to predict this new optimal gait and rapidly returned to it within a few steps if perturbed away. We then used our data to develop reinforcement learning models that can predict experimental behaviours, and applied these models to inductively reason about how the nervous system optimizes coordination. We conclude that the nervous system optimizes for energy using a prediction of the optimal gait, and then refines this prediction with the cost of each new walking step.

Energy Optimization During Walking Can Be a Primarily Implicit Process

“…To create a task where participants had to adapt their gait in order to minimize energy expenditure, we leveraged our previous paradigm where robotic exoskeletons are used to shift people's energetically optimal step frequency. We have previously shown, in a single-task context, that people adapt toward energy optimal step frequencies (Selinger et al 2015(Selinger et al , 2019.…”

“…-10% vs. -15%). Here and in other studies subsequent to the single-task experiment (Abram et al 2019;Selinger et al 2019;Simha et al 2019Simha et al , 2020 we have chosen to simplify our protocol to a single high and low tempo, which are used during both the Habituation Period and the Second Experience Period. Second, in the original single-task experiment, the Second Experience Period was 30 minutes in length, while here it is 40 minutes due to the addition of two extra metronome bouts.…”

“…To determine our minimum required participant number we performed an a priori power analysis for our primary outcome measure-step frequency adaptation. Based on our two previous studies (Selinger et al 2015(Selinger et al , 2019 we expected complete energy optimization to result in participants decreasing their step frequency by approximately 5% and with an across participant standard deviation of approximately 3.5%, when exposed to the penalize-high controller. To detect an across-participant average difference in step frequency of at least 5%, given an acrossparticipant average standard deviation in step frequency of 3.5% and a single-task participant number of 14, we calculated that we required a minimum of only four dual-task participants to achieve a power of 0.8.…”

“…In turn, by altering these coordination patterns we choose between different gaits, such as walking or running, and adapt countless gait parameters, such as speed, step frequency, and limb symmetry. Our research group, and others, have recently demonstrated that gait adaptations can be driven by continuous optimization of energy expenditure-when searching the expanse of possible gaits, we often prefer and converge on those that minimize the calories we burn in a given context (Abram et al 2019;Finley et al 2013;Roemmich et al 2019;Selinger et al 2015Selinger et al , 2019. However, whether energy optimization is primarily an implicit process-occurring automatically and with minimal cognitive attention-or rather an explicit process-occurring as a result of a conscious, attention-demanding, strategy-remains unclear (Frensch 1998;Kahneman and Egan 2011;Mazzoni and Krakauer 2006).…”

“…We define energy optimization, our primary task of interest, as the process of adapting one's gait to minimize metabolic energy expenditure. To study the energy optimization process, we leverage our previous experimental paradigm where robotic exoskeletons are used to shift people's energetically optimal step frequency to frequencies lower than normally preferred (Selinger et al 2015(Selinger et al , 2019. We evaluate performance in this task by adaptation toward the optima, measured by decreases in step frequency.…”