Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot

Ji, Yandong; Li, Zhongyu; Sun, Yujun; Peng, Xue Bin; Levine, Sergey; Berseth, Glen; Sreenath, Koushil

doi:10.1109/iros47612.2022.9981984

Cited by 35 publications

(12 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There have been efforts to obtain a multi-task policy, such as walking at different velocities using different gaits, conditioned only on variable commands [46], [53]- [55], which requires more extensive tuning due to the lack of a gait prior. Providing the robot with different reference motions for different skills can be helpful, but requires additional parameterization of the reference motions (e.g., a gait library) [45], [56]- [58], policy distillation [43], or a motion prior [59]- [61]. There is also a line of research to explicitly provide contact sequences for legged robots [7], [62]- [64].…”

Section: Related Workmentioning

confidence: 99%

“…3) Sim-to-Real Transfer for Legged Robots: To tackle sim-to-real transfer for RL-based methods, some works have sought to directly train policies directly in the real world [65]- [67], but most of the prior work, especially for dynamic skills, leverages a simulator to train the legged robot with extensive dynamics randomization [68] and then zeroshot transfer to the real world [7], [38], [39], [45], [52] or finetune with real-world data [16], [56], [69]. Since performing rollout on the hardware of human-scale bipedal robots is expensive, we use the zero-shot transfer method.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Li¹,

Peng²,

Abbeel³

et al. 2023

Preprint

View full text Add to dashboard Cite

0°-55°( a) (b) (c) Flight Phase Fig. 1: Representative dynamic jumping maneuvers performed by a bipedal robot Cassie using the proposed multi-task control policies. From left to right: (a) the robot jumps over 1.4 m and lands at the given target; (b) the robot jumps to a target that is 0.88 m in front of the robot and 0.44 m above the ground, and (c) the robot jumps in place while turning 55 • with a command to turn 60 • in place. The policies are trained in simulation and deployed on the hardware without further tuning. Video is at: https://youtu.be/aAPSZ2QFB-E.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Li¹,

Peng²,

Abbeel³

et al. 2023

Preprint

View full text Add to dashboard Cite

show abstract

“…Quadrupeds can also accurately kick a ball into a goal (Ji et al, 2022). This is an example of a contact-rich task, where MPC is not practical since the dynamics at play are difficult to model accurately (e.g., deformable contacts with the ball).…”

Section: Imparting An Impulse: Kickingmentioning

confidence: 99%

Legged robots for object manipulation: A review

et al. 2023

View full text Add to dashboard Cite

Legged robots can have a unique role in manipulating objects in dynamic, human-centric, or otherwise inaccessible environments. Although most legged robotics research to date typically focuses on traversing these challenging environments, many legged platform demonstrations have also included “moving an object” as a way of doing tangible work. Legged robots can be designed to manipulate a particular type of object (e.g., a cardboard box, a soccer ball, or a larger piece of furniture), by themselves or collaboratively. The objective of this review is to collect and learn from these examples, to both organize the work done so far in the community and highlight interesting open avenues for future work. This review categorizes existing works into four main manipulation methods: object interactions without grasping, manipulation with walking legs, dedicated non-locomotive arms, and legged teams. Each method has different design and autonomy features, which are illustrated by available examples in the literature. Based on a few simplifying assumptions, we further provide quantitative comparisons for the range of possible relative sizes of the manipulated object with respect to the robot. Taken together, these examples suggest new directions for research in legged robot manipulation, such as multifunctional limbs, terrain modeling, or learning-based control, to support a number of new deployments in challenging indoor/outdoor scenarios in warehouses/construction sites, preserved natural areas, and especially for home robotics.

show abstract

“…2) Legged Manipulation: Manipulation via locomotion has been studied in [22]- [26]. [24] proposed to use all the four legs of a quadrupedal robot as a dexterous manipulator by lying the robot back on the ground.…”

Section: Introductionmentioning

confidence: 99%

“…However this scarifies the mobility of the legged system and makes it no difference from a four finger manipulator. [26] enables the quadruped to kick a ball to a goal with real-world finetuning but is still a single skill result without any agility in locomotion and lacks generalization to long-horizon tasks.…”

Section: Introductionmentioning

confidence: 99%

Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Cheng¹,

Kumar²,

Pathak³

2023

Preprint

View full text Add to dashboard Cite

Locomotion has seen dramatic progress for walking or running across challenging terrains. However, robotic quadrupeds are still far behind their biological counterparts, such as dogs, which display a variety of agile skills and can use the legs beyond locomotion to perform several basic manipulation tasks like interacting with objects and climbing. In this paper, we take a step towards bridging this gap by training quadruped robots not only to walk but also to use the front legs to climb walls, press buttons, and perform object interaction in the real world. To handle this challenging optimization, we decouple the skill learning broadly into locomotion, which involves anything that involves movement whether via walking or climbing a wall, and manipulation, which involves using one leg to interact while balancing on the other three legs. These skills are trained in simulation using curriculum and transferred to the real world using our proposed sim2real variant that builds upon recent locomotion success. Finally, we combine these skills into a robust longterm plan by learning a behavior tree that encodes a highlevel task hierarchy from one clean expert demonstration. We evaluate our method in both simulation and real-world showing successful executions of both short as well as long-range tasks and how robustness helps confront external perturbations. Videos at https://robot-skills.github.io

show abstract

Hierarchical Reinforcement Learning for Precise Soccer Shooting Skills using a Quadrupedal Robot

Cited by 35 publications

References 32 publications

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Robust and Versatile Bipedal Jumping Control through Multi-Task Reinforcement Learning

Legged robots for object manipulation: A review

Legs as Manipulator: Pushing Quadrupedal Agility Beyond Locomotion

Contact Info

Product

Resources

About