Learning Agile Robotic Locomotion Skills by Imitating Animals

Peng, Xue Bin; Coumans, Erwin; Zhang, Tingnan; Lee, Tsang-Wei; Tan, Jie; Levine, Sergey

doi:10.48550/arxiv.2004.00784

Cited by 51 publications

(114 citation statements)

References 0 publications

Supporting

Mentioning

114

Contrasting

Order By: Relevance

“…Demonstrations have long been used in dealing with exploration issues in reinforcement learning for robotic tasks [11], [12], [13]. Recently, demonstrations have been used as one of the main ingredients to learn complicated locomotion behaviors in a DRL setting [14], [15]. In these works, a policy is trained using reinforcement learning to reproduce a given demonstration trajectory as well as possible in simulation.…”

Section: A Related Workmentioning

confidence: 99%

Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimization

Bogdanovic¹,

Khadiv²,

Righetti³

2021

Preprint

View full text Add to dashboard Cite

In this work we present a general, two-stage reinforcement learning approach for going from a single demonstration trajectory to a robust policy that can be deployed on hardware without any additional training. The demonstration is used in the first stage as a starting point to facilitate initial exploration. In the second stage, the relevant task reward is optimized directly and a policy robust to environment uncertainties is computed. We demonstrate and examine in detail performance and robustness of our approach on highly dynamic hopping and bounding tasks on a real quadruped robot (accompanying videos can be seen here).

show abstract

Section: A Related Workmentioning

confidence: 99%

Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimization

Bogdanovic¹,

Khadiv²,

Righetti³

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Our work is inspired by recent methods for animated character control via RL-based imitation of motion capture data [23]- [25]. Motion capture data has also been used to transfer dog behaviors to a real quadruped platform [10] or to provide adversarially learned motion priors [26]. Holden et al [27] use terrains that were fitted to existing motion capture trajectories for kinematic replay.…”

Section: Character Animation and Mocap Trackingmentioning

confidence: 99%

“…All units are centimeters. The spaces separating each pair of steps laterally were sampled from (10,15), the vertical offset and longitudinal offset of each step from (−5, 5), the length of the gaps in the longitudinal direction from (15,20) and the length and the width of the steps of each pair from (20, 40) and (30, 60) respectively. The steps were 72 cm high and we applied random rotations on the pitch and roll axes with values sampled from (−0.15, 0.15) radians.…”

Section: Wavy Stepsmentioning

confidence: 99%

“…One way to circumvent exploration problems and also prevent undesirable gaits, is through imitation learning. Motion capture data from a dog has been shown to be a useful source of behaviors from which natural looking gaits were successfully transferred to a quadruped robot [10]. However, obtaining motion capture data for a wide variety of terrains would be very costly.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner

Brakel¹,

Bohez²,

Hasenclever³

et al. 2021

Preprint

View full text Add to dashboard Cite

Dynamic quadruped locomotion over challenging terrains with precise foot placements is a hard problem for both optimal control methods and Reinforcement Learning (RL). Non-linear solvers can produce coordinated constraint satisfying motions, but often take too long to converge for online application. RL methods can learn dynamic reactive controllers but require carefully tuned shaping rewards to produce good gaits and can have trouble discovering precise coordinated movements. Imitation learning circumvents this problem and has been used with motion capture data to extract quadruped gaits for flat terrains. However, it would be costly to acquire motion capture data for a very large variety of terrains with height differences. In this work, we combine the advantages of trajectory optimization and learning methods and show that terrain adaptive controllers can be obtained by training policies to imitate trajectories that have been planned over procedural terrains by a non-linear solver. We show that the learned policies transfer to unseen terrains and can be fine-tuned to dynamically traverse challenging terrains that require precise foot placements and are very hard to solve with standard RL.

show abstract

“…The capability of blind walking, i.e., by using propioceptive feedback only, has achieved remarkable performance using either mixed-integer nonlinear optimization [1], [2] and mathematical model-based control [3]- [6], or modelfree learning which heavily utilizes physics simulations to train control policies by reinforcement learning [7]- [12] or imitation learning [13]- [15].…”

Section: Introductionmentioning

confidence: 99%

Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations

Acero¹,

Yu²,

Li³

2021

Preprint

View full text Add to dashboard Cite

Legged robots have achieved remarkable performance in blind walking using either model-based control or data-driven deep reinforcement learning. To proactively navigate and traverse various terrains, active use of visual perception becomes indispensable, and this work aims to exploit the use of sparse visual observations to achieve perceptual locomotion over a range of commonly seen bumps, ramps, and stairs in human-centred environments. We first formulate the selection of minimal visual input that can represent the uneven surfaces of interest, and propose a learning framework that integrates such exteroceptive and proprioceptive data. We specifically select state observations and design a training curriculum to learn feedback control policies more effectively over a range of different terrains. Using an extensive benchmark, we validate the learned policy in tasks that require omnidirectional walking over flat ground and forward locomotion over terrains with obstacles, showing a high success rate of traversal. Particularly, the robot performs autonomous perceptual locomotion with minimal visual perception using depth measurements, which are easily available from a Lidar or RGB-D sensor, and successfully demonstrates robust ascent and descent over high stairs of 20 cm step height, i.e., 50% of its leg length. Video summary can be found at https://youtu.be/vtp43jYQ5w4.

show abstract

Learning Agile Robotic Locomotion Skills by Imitating Animals

Cited by 51 publications

References 0 publications

Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimization

Model-free Reinforcement Learning for Robust Locomotion using Demonstrations from Trajectory Optimization

Learning Coordinated Terrain-Adaptive Locomotion by Imitating a Centroidal Dynamics Planner

Learning Perceptual Locomotion on Uneven Terrains using Sparse Visual Observations

Contact Info

Product

Resources

About