Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robots

Abstract

Legged locomotion is commonly studied and expressed as a discrete set of gait patterns, like walk, trot, gallop, which are usually treated as given and pre-programmed in legged robots for efficient locomotion at different speeds. However, fixing a set of pre-programmed gaits limits the generality of locomotion. Recent animal motor studies show that these conventional gaits are only prevalent in ideal flat terrain conditions while real-world locomotion is unstructured and more like bouts of intermittent steps. What principles could lead to both structured and unstructured patterns across mammals and how to synthesize them in robots? In this work, we take an analysis-by-synthesis approach and learn to move by minimizing mechanical energy. We demonstrate that learning to minimize energy consumption plays a key role in the emergence of natural locomotion gaits at different speeds in real quadruped robots. The emergent gaits are structured in ideal terrains and look similar to that of horses and sheep. The same approach leads to unstructured gaits in rough terrains which is consistent with the findings in animal motor control. We validate our hypothesis in both simulation and real hardware across natural terrains. Videos at https://energy-locomotion.github.io.

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

“…Since it is challenging in general to learn a single policy with RL to perform various tasks [48], many prior works focus on learning a single-task policy [16], [17], [49], [50] for legged robots, such as just forward walking [39], [51], [52]. There have been efforts to obtain a multi-task policy, such as walking at different velocities using different gaits, conditioned only on variable commands [46], [53]- [55], which requires more extensive tuning due to the lack of a gait prior. Providing the robot with different reference motions for different skills can be helpful, but requires additional parameterization of the reference motions (e.g., a gait library) [45], [56]- [58], policy distillation [43], or a motion prior [59]- [61].…”

Rapid Locomotion via Reinforcement Learning

“…The mechanical design of the ANYmal robot is thought to limit it from running at higher speeds. [13,23] investigated the capability of model-free controllers to efficiently traverse diverse terrains on the Unitree A1, a small robot with similar size, actuation, and cost to the Mini Cheetah. Although the A1's built-in MPC controller has a maximum running speed of 3.3 m/s, these learning-based works only demonstrated the robot running up to maximum speed 1.8 m/s.…”

The Introspective Agent: Interdependence of Strategy, Physiology, and Sensing for Embodied Agents

“…While a higher planning ability may contribute to a more successful predator and therefore a higher gain in energy, it is very energetically costly to run these systems. Additionally, prior work in robotics has shown that under different scenarios, systems will find different optimal policies to minimize energy consumption [12].…”