Learning robust perceptive locomotion for quadrupedal robots in the wild

Miki, Takahiro; Lee, Joonho; Hwangbo, Jemin; Wellhausen, Lorenz; Koltun, Vladlen; Hutter, Marco

doi:10.1126/scirobotics.abk2822

Cited by 409 publications

(316 citation statements)

References 58 publications

Supporting

Mentioning

314

Contrasting

Order By: Relevance

“…We spawn 4096 environments in parallel to learn all three tasks simultaneously in a single neural network. The number of environments per task is weighted according to their approximate difficulty, e.g., [1,1,5] in the case of the tasks described above. The state-transitions collected during the roll-outs of these environments are mapped using a function φ(s) such that it extracts the linear and angular base velocity, gravity direction in base frame, the base's height above ground, joint position and velocity, and finally the position of the wheels relative to the robot's base-frame, i.e., φ(s) = ( ẋbase , x z , e base , q, q, x ee,base ) ∈ R 50 .…”

Section: Resultsmentioning

confidence: 99%

See 1 more Smart Citation

Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning

Vollenweider¹,

Bjelonic²,

Klemm³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

In recent years, reinforcement learning (RL) has shown outstanding performance for locomotion control of highly articulated robotic systems. Such approaches typically involve tedious reward function tuning to achieve the desired motion style. Imitation learning approaches such as adversarial motion priors aim to reduce this problem by encouraging a pre-defined motion style. In this work, we present an approach to augment the concept of adversarial motion prior-based RL to allow for multiple, discretely switchable styles. We show that multiple styles and skills can be learned simultaneously without notable performance differences, even in combination with motion data-free skills. Our approach is validated in several real-world experiments with a wheeled-legged quadruped robot showing skills learned from existing RL controllers and trajectory optimization, such as ducking and walking, and novel skills such as switching between a quadrupedal and humanoid configuration. For the latter skill, the robot is required to stand up, navigate on two wheels, and sit down. Instead of tuning the sit-down motion, we verify that a reverse playback of the stand-up movement helps the robot discover feasible sit-down behaviors and avoids tedious reward function tuning.

show abstract

Section: Resultsmentioning

confidence: 99%

“…Reinforcement Learning (RL) had a significant impact in the space of legged locomotion, showcasing robust policies that can handle a wide variety of challenging terrain in the real world [1]. With this advancement, we believe that these articulated robots can perform specialized motions like their natural counterparts.…”

Section: Introductionmentioning

confidence: 99%

Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning

Vollenweider¹,

Bjelonic²,

Klemm³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…OpenAI trained Dactyl for a human-like robot hand to dextrously manipulate physical objects (OpenAI et al, 2018). A series of work has advanced the progress in quadrupedal robots locomotion over challenging terrains in the wild, integrating both exteroceptive and proprioceptive perception (Hwangbo et al, 2019;Lee et al, 2020;Miki et al, 2022). Peng et al (2018) present DeepMimic for simulated humanoid to perform highly dynamic and acrobatic skills.…”

Section: Roboticsmentioning

confidence: 99%

Reinforcement Learning in Practice: Opportunities and Challenges

Li¹

2022

Preprint

View full text Add to dashboard Cite

This article is a gentle discussion about the field of reinforcement learning for real life, about opportunities and challenges, with perspectives and without technical details, touching a broad range of topics. The article is based on both historical and recent research papers, surveys, tutorials, talks, blogs, and books. Various groups of readers, like researchers, engineers, students, managers, investors, officers, and people wanting to know more about the field, may find the article interesting.In this article, we first give a brief introduction to reinforcement learning (RL), and its relationship with deep learning, machine learning and AI. Then we discuss opportunities of RL, in particular, applications in products and services, games, recommender systems, robotics, transportation, economics and finance, healthcare, education, combinatorial optimization, computer systems, and science and engineering. The we discuss challenges, in particular, 1) foundation, 2) representation, 3) reward, 4) model, simulation, planning, and benchmarks, 5) learning to learn a.k.a. meta-learning, 6) off-policy/offline learning, 7) software development and deployment, 8) business perspectives, and 9) more challenges. We conclude with a discussion, attempting to answer: "Why has RL not been widely adopted in practice yet?" and "When is RL helpful?".

show abstract

“…Sensory information from cameras and LiDAR reveals a great deal about the characteristics of terrain, and can be leveraged to anticipate its effects on the robot's dynamics. While researchers have recently started to incorporate visual information into gait planners for legged-robots [13], wheeled mobile robot motion planners that use visual information have been mostly limited to end-to-end learning solutions.…”

Section: B Error Modelling and Reactive Controlmentioning

confidence: 99%

VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics

Karnan¹,

Sikand²,

Atreya³

et al. 2022

Preprint

View full text Add to dashboard Cite

One of the key challenges in high-speed off-road navigation on ground vehicles is that the kinodynamics of the vehicle-terrain interaction can differ dramatically depending on the terrain. Previous approaches to addressing this challenge have considered learning an inverse kinodynamics (IKD) model, conditioned on inertial information of the vehicle to sense the kinodynamic interactions. In this paper, we hypothesize that to enable accurate high-speed off-road navigation using a learned IKD model, in addition to inertial information from the past, one must also anticipate the kinodynamic interactions of the vehicle with the terrain in the future. To this end, we introduce Visual-Inertial Inverse Kinodynamics (VI-IKD), a novel learning based IKD model that is conditioned on visual information from a terrain patch ahead of the robot in addition to past inertial information, enabling it to anticipate kinodynamic interactions in the future. We validate the effectiveness of VI-IKD in accurate high-speed off-road navigation experimentally on a scale 1/5 UT-AlphaTruck off-road autonomous vehicle in both indoor and outdoor environments and show that compared to other state-of-the-art approaches, VI-IKD enables more accurate and robust off-road navigation on a variety of different terrains at speeds of up to 3.5m/s.

show abstract

Learning robust perceptive locomotion for quadrupedal robots in the wild

Cited by 409 publications

References 58 publications

Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning

Advanced Skills through Multiple Adversarial Motion Priors in Reinforcement Learning

Reinforcement Learning in Practice: Opportunities and Challenges

VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics

Contact Info

Product

Resources

About