Rapid Locomotion via Reinforcement Learning

Margolis, Gabriel B.; Yang, Ge; Paigwar, Kartik; Chen, Tao; Agrawal, Pulkit

doi:10.48550/arxiv.2205.02824

Cited by 8 publications

(21 citation statements)

References 36 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Training on hardware can lead to better performance, but the range of behaviors that can be learned has so far been limited because of safety and data efficiency concerns. Similar to our work, prior work has shown that learned gaits can achieve higher velocities compared with scripted gaits (49)(50)(51). However, the gaits have been specifically trained to attain high speeds instead of emerging as a result of optimizing for a higher level goal.…”

Section: Discussionmentioning

confidence: 77%

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Haarnoja,

Moran,

Lever

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent’s tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

show abstract

Section: Discussionmentioning

confidence: 77%

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Haarnoja,

Moran,

Lever

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

show abstract

“…Another approach to generating running and jumping controllers is through deep reinforcement learning. For example, fast trot-running [16], [17] and bounding [18] have autonomously emerged end-to-end through learning frameworks. Advanced skills can also be learned by incorporating terrain-awareness for tasks such as climbing and jumping gaps [19], or rough terrain locomotion in the wild [20].…”

Section: Recent Work Extends Such Single Jumps With Model Predictivementioning

confidence: 99%

CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion

Bellegarda

Ijspeert

2022

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

In this letter, we present a method for integrating central pattern generators (CPGs), i.e. systems of coupled oscillators, into the deep reinforcement learning (DRL) framework to produce robust and omnidirectional quadruped locomotion. The agent learns to directly modulate the intrinsic oscillator setpoints (amplitude and frequency) and coordinate rhythmic behavior among different oscillators. This approach also allows the use of DRL to explore questions related to neuroscience, namely the role of descending pathways, interoscillator couplings, and sensory feedback in gait generation. We train our policies in simulation and perform a sim-to-real transfer to the Unitree A1 quadruped, where we observe robust behavior to disturbances unseen during training, most notably to a dynamically added 13.75 kg load representing 115% of the nominal quadruped mass. We test several different observation spaces based on proprioceptive sensing and show that our framework is deployable with no domain randomization and very little feedback, where along with the oscillator states, it is possible to provide only contact booleans in the observation space. Video results can be found at https://youtu.be/xqXHLzLsEV4.

show abstract

“…It can also build large-scale realistic complex scenes, and its underlying PhysX engine can accurately and realistically model and simulate the motion of objects. Therefore, more researchers have begun to use Isaac Gym as the implementation and verification platform of DRL algorithm [35][36][37][38] .…”

Section: Simulatormentioning

confidence: 99%

Deep reinforcement learning for real-world quadrupedal locomotion: a comprehensive review

Zhang¹,

Li²,

Wang³

2022

Intell Robot

View full text Add to dashboard Cite

Building controllers for legged robots with agility and intelligence has been one of the typical challenges in the pursuit of artificial intelligence (AI). As an important part of the AI field, deep reinforcement learning (DRL) can realize sequential decision making without physical modeling through end-to-end learning and has achieved a series of major breakthroughs in quadrupedal locomotion research. In this review article, we systematically organize and summarize relevant important literature, covering DRL algorithms from problem setting to advanced learning methods. These algorithms alleviate the specific problems encountered in the practical application of robots to a certain extent. We first elaborate on the general development trend in this field from several aspects, such as the DRL algorithms, simulation environments, and hardware platforms. Moreover, core components in the algorithm design, such as state and action spaces, reward functions, and solutions to reality gap problems, are highlighted and summarized. We further discuss open problems and propose promising future research directions to discover new areas of research.

show abstract

Rapid Locomotion via Reinforcement Learning

Cited by 8 publications

References 36 publications

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

CPG-RL: Learning Central Pattern Generators for Quadruped Locomotion

Deep reinforcement learning for real-world quadrupedal locomotion: a comprehensive review

Contact Info

Product

Resources

About