Research on Foothold Optimization of the Quadruped Crawling Robot based on Reinforcement Learning

Abstract

Background:: Quadruped crawling robots will be faced with stability problems when walking on a raised slope. The stability of robot is affected by gait planning and selection of its foothold in this terrain. The slope reaction force on anterior and posterior legs is uneven. The selection strategy of its foothold should achieve good performance for the stability of the quadruped crawling robot. background: Quadruped crawling robots will be faced with stability problems when walking on the raised slope. The stability of robot is affected by gait planning and selection of its foothold in this terrain. The slope reaction force on anterior and posterior legs is uneven. The selection strategy of its foothold should achieve good performance for the stability of quadruped crawling robot. Objective:: Aimed at the uneven problem of slope reaction force on the anterior and posterior legs of the quadruped crawling robot when walking on the raised slope, a patent method for foothold optimization using reinforcement learning based on strategy search is proposed. objective: Aimed at the uneven problem of slope reaction force on anterior and posterior legs of the quadruped crawling robot when walking on the raised slope, a reinforcement learning method based on strategy search is adopted to select its foothold for quadruped crawling robot. Methods:: The kinematic model of the quadruped crawling robot is created in D-H coordinate method. According to the gait timing sequence method, the frame description of the quadruped crawling robot's gait on the slope is proposed. The fitting polynomial coefficients and fitting curves of all joints of the leg can be obtained by using the polynomial fitting calculation method. The reinforcement learning method based on Q-learning algorithm is proposed to find the optimal foothold by interacting with the slope environment. Comparative simulation and test of other gait and climbing slope gait, the climbing slope gait with and without the Q-learning algorithm is carried out by MATLAB platform. Results:: When the quadruped crawling robot adopts the reinforcement learning method based on Qlearning algorithm to select foothold, the robot posture curves are compared without optimization strategy. The result proves that the selection strategy of its foothold is valid. Conclusion:: The selection strategy of its foothold with reinforcement learning based on Q-learning algorithm can improve the stability of the quadruped crawling robot on the raised sloped. other: not have