Emergence of human-comparable balancing behaviours by deep reinforcement learning

Yang, Chuanyu; Komura, Taku; Li, Zhibin

doi:10.1109/humanoids.2017.8246900

Cited by 16 publications

(12 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The total reward is composed of a weighted sum of scalar components i ω i r i , where r i is the reward term and w i its weight. In order to provide a similar scale for each of them, and therefore improving the interpretability of the total reward, we process the real and vector components with a Radial Basis Function (RBF) kernel [41] with a dimension given by a cutoff parameter calculated from the desired sensitivity. Appendix A provides a more detailed description of the kernel.…”

Section: Rewardmentioning

confidence: 99%

On the Emergence of Whole-Body Strategies From Humanoid Robot Push-Recovery Learning

Ferigo

Camoriano

Viceconte

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Balancing and push-recovery are essential capabilities enabling humanoid robots to solve complex locomotion tasks. In this context, classical control systems tend to be based on simplified physical models and hard-coded strategies. Although successful in specific scenarios, this approach requires demanding tuning of parameters and switching logic between specifically-designed controllers for handling more general perturbations. We apply model-free Deep Reinforcement Learning for training a general and robust humanoid push-recovery policy in a simulation environment. Our method targets high-dimensional whole-body humanoid control and is validated on the iCub humanoid. Reward components incorporating expert knowledge on humanoid control enable fast learning of several robust behaviors by the same policy, spanning the entire body. We validate our method with extensive quantitative analyses in simulation, including out-of-sample tasks which demonstrate policy robustness and generalization, both key requirements towards real-world robot deployment.

show abstract

Section: Rewardmentioning

confidence: 99%

On the Emergence of Whole-Body Strategies From Humanoid Robot Push-Recovery Learning

Ferigo

Camoriano

Viceconte

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…The learning task JPL and WSC were inspired by similar works [Yang et al 2017, Yang et al 2018 where the simulated robot was periodically pushed while trying to keep upright. In this work we faced a new challenge: trying to learn a Push Recovery policy while the robot runs.…”

Section: Running Agent With No Priors -Ranpmentioning

confidence: 99%

Learning Push Recovery Strategies for Bipedal Walking

Melo¹,

Máximo²,

Cunha³

2021

Anais Estendidos Do XIII Simpósio Brasileiro De Robótica E XVIII Simpósio Latino Americano De Robótica (SBR/LARS Estendido 2021

View full text Add to dashboard Cite

The present work provides an implementation of a Push Recovery controller that aids the walking engine used by a humanoid simulated robot. The simulation environment is the Robocup Soccer 3D Simulation League. The learned movement policies exceeded our original walking engine. In addition, we evaluated the policies and detected undesired biases. New methodologies were introduced in order to eliminate it.

show abstract

“…The design of the reward function is a crucial part in reinforcement learning as the reward governs the outcome behavior. The reward design follows a similar design rule as in [29]. Balancing can be divided into four subtasks: regulating upper body pose, regulating CoM position, regulating CoM velocity, and regulating ground contact force.…”

Section: Design Of Reward Functionmentioning

confidence: 99%

Learning Whole-Body Motor Skills for Humanoids

Yang

Yuan

Merkt

et al. 2018

2018 IEEE-RAS 18th International Conference on Humanoid Robots (Humanoids)

Self Cite

View full text Add to dashboard Cite

This paper presents a hierarchical framework for Deep Reinforcement Learning that acquires motor skills for a variety of push recovery and balancing behaviors, i.e., ankle, hip, foot tilting, and stepping strategies. The policy is trained in a physics simulator with realistic setting of robot model and low-level impedance control that are easy to transfer the learned skills to real robots. The advantage over traditional methods is the integration of high-level planner and feedback control all in one single coherent policy network, which is generic for learning versatile balancing and recovery motions against unknown perturbations at arbitrary locations (e.g., legs, torso). Furthermore, the proposed framework allows the policy to be learned quickly by many state-of-the-art learning algorithms. By comparing our learned results to studies of preprogrammed, special-purpose controllers in the literature, self-learned skills are comparable in terms of disturbance rejection but with additional advantages of producing a wide range of adaptive, versatile and robust behaviors.

show abstract

Emergence of human-comparable balancing behaviours by deep reinforcement learning

Cited by 16 publications

References 21 publications

On the Emergence of Whole-Body Strategies From Humanoid Robot Push-Recovery Learning

On the Emergence of Whole-Body Strategies From Humanoid Robot Push-Recovery Learning

Learning Push Recovery Strategies for Bipedal Walking

Learning Whole-Body Motor Skills for Humanoids

Contact Info

Product

Resources

About