Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Li, Zhongyu; Peng, Xue Bin; Abbeel, Pieter; Levine, Sergey; Berseth, Glen; Sreenath, Koushil

doi:10.48550/arxiv.2302.09450

Cited by 7 publications

(11 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our work similarly relies on zero-shot sim-to-real transfer and model randomization but instead focuses on a range of dynamic motions, stability, long horizon tasks, object manipulation, and multiagent competitive play. Most recent work in this area relies on some form of sim-to-real transfer (14,15,17,19,27,(40)(41)(42), which can help to reduce the safety and data efficiency concerns associated with training directly on hardware. A common theme is that an unexpectedly small number of techniques can be sufficient to reduce the sim-to-real gap (37,43), which is also supported by our results.…”

Section: Discussionmentioning

confidence: 99%

“…Quadrupedal platforms constitute most legged locomotion research, but an increasing number of works consider bipedal platforms. Recent works have produced behaviors including walking and running (24,25), stair climbing (26), and jumping (27). Most recent works have focused on high-quality, full-sized bipeds and humanoids, with a much smaller number (48,(52)(53)(54) targeting more basic platforms whose simpler and less precise actuators and sensors pose additional challenges in terms of sim-to-real transfer.…”

Section: Discussionmentioning

confidence: 99%

“…On the other hand, much less work has been dedicated to the control of humanoids and bipeds, which impose additional challenges around stability, robot safety, number of degrees of freedom, and availability of suitable hardware. The existing learning-based work has been more limited and focused on learning and transfer of distinct basic skills, such as walking (24), running (25), stair climbing (26), and jumping (27). The state of the art in humanoid control uses targeted model-based predictive control (28), thus limiting the generality of the method.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Haarnoja,

Moran,

Lever

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

We investigated whether deep reinforcement learning (deep RL) is able to synthesize sophisticated and safe movement skills for a low-cost, miniature humanoid robot that can be composed into complex behavioral strategies. We used deep RL to train a humanoid robot to play a simplified one-versus-one soccer game. The resulting agent exhibits robust and dynamic movement skills, such as rapid fall recovery, walking, turning, and kicking, and it transitions between them in a smooth and efficient manner. It also learned to anticipate ball movements and block opponent shots. The agent’s tactical behavior adapts to specific game contexts in a way that would be impractical to manually design. Our agent was trained in simulation and transferred to real robots zero-shot. A combination of sufficiently high-frequency control, targeted dynamics randomization, and perturbations during training enabled good-quality transfer. In experiments, the agent walked 181% faster, turned 302% faster, took 63% less time to get up, and kicked a ball 34% faster than a scripted baseline.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Haarnoja,

Moran,

Lever

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

show abstract

“…falls (14,15), and dribbling with a football (16,17). In parallel, bipedal robots have also demonstrated their agile capabilities by walking blindly on rough terrain (18) and jumping on obstacles (19).…”

Section: Of 14mentioning

confidence: 99%

ANYmal parkour: Learning agile navigation for quadrupedal robots

Hoeller,

Rudin,

Sako

et al. 2024

Sci. Robot.

View full text Add to dashboard Cite

Performing agile navigation with four-legged robots is a challenging task because of the highly dynamic motions, contacts with various parts of the robot, and the limited field of view of the perception sensors. Here, we propose a fully learned approach to training such robots and conquer scenarios that are reminiscent of parkour challenges. The method involves training advanced locomotion skills for several types of obstacles, such as walking, jumping, climbing, and crouching, and then using a high-level policy to select and control those skills across the terrain. Thanks to our hierarchical formulation, the navigation policy is aware of the capabilities of each skill, and it will adapt its behavior depending on the scenario at hand. In addition, a perception module was trained to reconstruct obstacles from highly occluded and noisy sensory data and endows the pipeline with scene understanding. Compared with previous attempts, our method can plan a path for challenging scenarios without expert demonstration, offline computation, a priori knowledge of the environment, or taking contacts explicitly into account. Although these modules were trained from simulated data only, our real-world experiments demonstrate successful transfer on hardware, where the robot navigated and crossed consecutive challenging obstacles with speeds of up to 2 meters per second.

show abstract

“…Robotic quadrupeds and bipeds can walk across challenging scenarios ranging from hiking on hills to walking over rocky surfaces near river beds [1]- [11]. What is next for legged systems?…”

Section: Introductionmentioning

confidence: 99%

Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Cheng¹,

Kumar²,

Pathak³

2023

Preprint

View full text Add to dashboard Cite

Locomotion has seen dramatic progress for walking or running across challenging terrains. However, robotic quadrupeds are still far behind their biological counterparts, such as dogs, which display a variety of agile skills and can use the legs beyond locomotion to perform several basic manipulation tasks like interacting with objects and climbing. In this paper, we take a step towards bridging this gap by training quadruped robots not only to walk but also to use the front legs to climb walls, press buttons, and perform object interaction in the real world. To handle this challenging optimization, we decouple the skill learning broadly into locomotion, which involves anything that involves movement whether via walking or climbing a wall, and manipulation, which involves using one leg to interact while balancing on the other three legs. These skills are trained in simulation using curriculum and transferred to the real world using our proposed sim2real variant that builds upon recent locomotion success. Finally, we combine these skills into a robust longterm plan by learning a behavior tree that encodes a highlevel task hierarchy from one clean expert demonstration. We evaluate our method in both simulation and real-world showing successful executions of both short as well as long-range tasks and how robustness helps confront external perturbations. Videos at https://robot-skills.github.io

show abstract

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Cited by 7 publications

References 58 publications

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

Learning agile soccer skills for a bipedal robot with deep reinforcement learning

ANYmal parkour: Learning agile navigation for quadrupedal robots

Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Contact Info

Product

Resources

About