Decoding Motor Skills of Artificial Intelligence and Human Policies: A Study on Humanoid and Human Balance Control

Yuan, Kai; McGreavy, Christopher; Yang, Chuanyu; Wolfslag, Wouter; Li, Zhibin

doi:10.1109/mra.2020.2980547

Cited by 11 publications

(6 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lastly, Yan et al [3] delve into the stability of humanoid robots constructed using cutting-edge machine learning and AI techniques. A deep reinforcement learning-based control framework, meticulously crafted to empower humanoid robots with the dexterity needed for push recovery, takes center stage.…”

Section: Human-inspired Approachesmentioning

confidence: 99%

See 1 more Smart Citation

Humanoid Robot Motion Planning Approaches: a Survey

de Lima,

Khan,

Tufail

et al. 2024

J Intell Robot Syst

View full text Add to dashboard Cite

Humanoid robots are complex, dynamic systems. Any humanoid robotic application starts with determining a sequence of optimal paths to perform a given task in a known or unknown environment. This paper critically reviews and rates available literature on the three key areas of multi-level motion and task planning for humanoid robots. First is efficiency while navigating and manipulating objects in environments designed for humans. Here, the research has broadly been summarized as behavior cloning approaches. Second is robustness to perturbations and collisions caused by operation in dynamic and unpredictable environments. Here, the modeling approaches integrated into motion planning algorithms have been the focus of many researchers studying humanoid motion’s balance and dynamic stability aspects. Last is real-time performance, wherein the robot must adjust its motion based on the most recent sensory data to achieve the required degree of interaction and responsiveness. Here, the focus has been on the kinematic constraints imposed by the robot’s mechanical structure and joint movements. The iterative nature of solving constrained optimization problems, the computational complexity of forward and inverse kinematics, and the requirement to adjust to a rapidly changing environment all pose challenges to real-time performance. The study has identified current trends and, more importantly, research gaps while pointing to areas needing further investigation.

show abstract

Section: Human-inspired Approachesmentioning

confidence: 99%

“…Cloning approaches have been discussed in Sect. 3. Inverted pendulum-type schemes are discussed in Sect.…”

Section: Introductionmentioning

confidence: 99%

Humanoid Robot Motion Planning Approaches: a Survey

de Lima,

Khan,

Tufail

et al. 2024

J Intell Robot Syst

View full text Add to dashboard Cite

show abstract

“…Upright standing offers a large variety of recovery strategies that can be leveraged in case of emergency to avoid falling down, among them: ankle, hip, stepping, height modulation and foot-tilting for any legged robot, plus angular momentum modulation for humanoid robots [13]. For small perturbations, in-place recovery strategies controlling the Center of Pressure (CoP) [14], the centroidal angular momentum [15], or using foot-tilting [16], [17] are sufficient.…”

Section: A Classical Non-linear Controlmentioning

confidence: 99%

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

Duburcq¹,

Schramm²,

Boéris³

et al. 2022

Preprint

View full text Add to dashboard Cite

State-of-the-art reinforcement learning is now able to learn versatile locomotion, balancing and push-recovery capabilities for bipedal robots in simulation. Yet, the reality gap has mostly been overlooked and the simulated results hardly transfer to real hardware. Either it is unsuccessful in practice because the physics is over-simplified and hardware limitations are ignored, or regularity is not guaranteed and unexpected hazardous motions can occur. This paper presents a reinforcement learning framework capable of learning robust standing push recovery for bipedal robots with a smooth out-of-the-box transfer to reality, requiring only instantaneous proprioceptive observations. By combining original termination conditions and policy smoothness conditioning, we achieve stable learning, simto-real transfer and safety using a policy without memory nor observation history. Reward shaping is then used to give insights into how to keep balance. We demonstrate its performance in reality on the lower-limb medical exoskeleton Atalante.

show abstract

“…We analysed the diversity of skills of MELA and MoE by a t-distributed Stochastic Neighbor Embedding (t-SNE) analysis ( Maaten and Hinton, 2008 ). T-SNE projects high-dimensionals NN activation on a 2D plane by clustering similar NN activations together but keeping dissimilar data points distant, which can be used to analyse robotic behaviours ( Yuan et al, 2020 ).…”

Section: Comparison Of Multi-expert Learning Architecture and Mixture...mentioning

confidence: 99%

Multi-expert synthesis for versatile locomotion and manipulation skills

Yuan

2022

Front. Robot. AI

Self Cite

View full text Add to dashboard Cite

This work focuses on generating multiple coordinated motor skills for intelligent systems and studies a Multi-Expert Synthesis (MES) approach to achieve versatile robotic skills for locomotion and manipulation. MES embeds and uses expert skills to solve new composite tasks, and is able to synthesise and coordinate different and multiple skills smoothly. We proposed essential and effective design guidelines for training successful MES policies in simulation, which were deployed on both floating- and fixed-base robots. We formulated new algorithms to systematically determine task-relevant state variables for each individual experts which improved robustness and learning efficiency, and an explicit enforcement objective to diversify skills among different experts. The capabilities of MES policies were validated in both simulation and real experiments for locomotion and bi-manual manipulation. We demonstrated that the MES policies achieved robust locomotion on the quadruped ANYmal by fusing the gait recovery and trotting skills. For object manipulation, the MES policies learned to first reconfigure an object in an ungraspable pose and then grasp it through cooperative dual-arm manipulation.

show abstract

Decoding Motor Skills of Artificial Intelligence and Human Policies: A Study on Humanoid and Human Balance Control

Cited by 11 publications

References 15 publications

Humanoid Robot Motion Planning Approaches: a Survey

Humanoid Robot Motion Planning Approaches: a Survey

Reactive Stepping for Humanoid Robots using Reinforcement Learning: Application to Standing Push Recovery on the Exoskeleton Atalante

Multi-expert synthesis for versatile locomotion and manipulation skills

Contact Info

Product

Resources

About