Hybrid Bipedal Locomotion Based on Reinforcement Learning and Heuristics

Abstract

Locomotion control has long been vital to legged robots. Agile locomotion can be implemented through either model-based controller or reinforcement learning. It is proven that robust controllers can be obtained through model-based methods and learning-based policies have advantages in generalization. This paper proposed a hybrid framework of locomotion controller that combines deep reinforcement learning and simple heuristic policy and assigns them to different activation phases, which provides guidance for adaptive training without producing conflicts between heuristic knowledge and learned policies. The training in simulation follows a step-by-step stochastic curriculum to guarantee success. Domain randomization during training and assistive extra feedback loops on real robot are also adopted to smooth the transition to the real world. Comparison experiments are carried out on both simulated and real Wukong-IV humanoid robots, and the proposed hybrid approach matches the canonical end-to-end approaches with higher rate of success, faster converging speed, and 60% less tracking error in velocity tracking tasks.

Realization of a Human-like Gait for a Bipedal Robot Based on Gait Analysis

“…The main walking control methods for the bipedal robots are model-based control and learning control. In this study, we use learning control, which has potential for future development and has been actively studied [23][24][25][26].…”

Running Gait and Control of Quadruped Robot Based on SLIP Model

“…This model serves as a mechanism model for single-leg control of running robots and compared to the LIP (Linear Inverted Pendulum) model, is more closely related to the elastic muscle-rich mammalian leg. It can explain the buffering mechanism for impact forces at the foot–ground contact [ 18 ]. Building on the SLIP model, Raibert developed the well-known heuristic three-part controller by fully utilizing its self-stabilization properties [ 19 , 20 ].…”

Design and Analysis of a Novel Omnidirectional Step-climbing Robot

“…(21)(22)(23) Additionally, in unfamiliar terrain, the balancing control problem during dynamic stepping is complex. (24)(25)(26) Recent research has focused on improving the motion speed of legged robots. (27,28) MRs with multiwheel or track structures have been found to be effective for assisting robots in climbing steps.…”