Reinforcement learning and neural network-based artificial intelligence control algorithm for self-balancing quadruped robot

Adaptive Quadruped Balance Control for Dynamic Environments Using Maximum-Entropy Reinforcement Learning

“…Moreover, if the tilt angle of external disturbance or uncontrolled robot is known, we can use the corresponding balance efficiency to estimate the tilt angle after adopting this policy to estimate whether it is worth trying to implement this policy on actual application conditions. can be replaced by and to calculate the balance efficiencies for robots that using the policies presented in this paper and [ 19 ]. …”

“…Unlike static environments, dynamic environments always cause various disturbances for robots. A recent work [ 19 ] observed that RL can be used to design the control algorithm for a quadruped robot to maintain balance in an unstable environment. In [ 19 ], RL was used to optimize a table-based deterministic policy in the finite discrete state and action spaces according to kinematic equations, i.e., the optimal actions were selected from 8 alternative actions through kinematic formulations when the quadruped system reached new states.…”

“…A recent work [ 19 ] observed that RL can be used to design the control algorithm for a quadruped robot to maintain balance in an unstable environment. In [ 19 ], RL was used to optimize a table-based deterministic policy in the finite discrete state and action spaces according to kinematic equations, i.e., the optimal actions were selected from 8 alternative actions through kinematic formulations when the quadruped system reached new states. Then an artificial neural network (ANN) is trained using the obtained pairs of states and actions through supervised learning to approximate the table-based policy, forming a continuous policy.…”

“…This paper proposes a convenient and adaptable approach to construct a self-balancing controller for quadruped robots. Our method employs RL and ANN for policy design, however, the design concept and process are thoroughly different from those in [ 19 ]. This work aims at learning self-balancing control policy during interaction with a simulated dynamic environment, and transferring the obtained policy to real robots, which abandons the construction of kinematic equations to simplify the design process and enhance the adaptivity of control policy.…”

“…As it is demonstrated that a challenging suite of training scenarios can help the trained policy succeed in a wide range of cases [ 14 , 18 ], we design an automatic changing disturbance curriculum to appropriately enhance the level of difficulty. A parameterized stochastic policy based on ANN is directly integrated into the RL process, other than using ANN to approximate a table-based deterministic policy that has already been obtained by RL [ 19 ]. The diversity of actions and exploration ability are ensured thanks to the employment of the changing disturbance curriculum, continuous spaces, a stochastic policy and a maximum-entropy RL framework.…”