Motion Planning for a Snake Robot using Double Deep Q-Learning

Khan, Semab Neimat; Mahmood, Tallat; Ullah, Syed Izzat; Ali, K. M. Akkas; Ullah, Anayat

doi:10.1109/icai52203.2021.9445200

Cited by 12 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve environmental adaptability of snake robots, a DRL-based framework with a double deep Q-learning-based technique is proposed to learn the optimal policy for reaching goal points in unknown environments in ref. [25]. However, main variables of different unknown environments are frictions and stiffness but not obstacles.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning-based motion control for snake robots in complex environments

Zhang,

Ju,

Cao

2024

Robotica

View full text Add to dashboard Cite

Snake robots can move flexibly due to their special bodies and gaits. However, it is difficult to plan their motion in multi-obstacle environments due to their complex models. To solve this problem, this work investigates a reinforcement learning-based motion planning method. To plan feasible paths, together with a modified deep Q-learning algorithm, a Floyd-moving average algorithm is proposed to ensure smoothness and adaptability of paths for snake robots’ passing. An improved path integral algorithm is used to work out gait parameters to control snake robots to move along the planned paths. To speed up the training of parameters, a strategy combining serial training, parallel training, and experience replaying modules is designed. Moreover, we have designed a motion planning framework consists of path planning, path smoothing, and motion planning. Various simulations are conducted to validate the effectiveness of the proposed algorithms.

show abstract

Section: Introductionmentioning

confidence: 99%

Reinforcement learning-based motion control for snake robots in complex environments

Zhang,

Ju,

Cao

2024

Robotica

View full text Add to dashboard Cite

show abstract

“…A simple sliding mode (SM) control theory was proposed for a differential drive‐enabled wheeled mobile robot (Dagci et al, 2013). To reach the goal point from a random start point, a double‐deep Q‐learning‐based technique to learn the optimal policy was proposed (Khan et al, 2021). To ensure the gait performs well in discrete action spaces and continuous state spaces (Shi et al, 2020), the Deep Q‐Network algorithm was employed to obtain a novel, efficient gait.…”

Section: Introductionmentioning

confidence: 99%

Modeling of novel circular gait motion through daisy sequence fitting algorithm in a vertical climbing snake robot

Megalingam,

Senthil,

Raghavan

et al. 2023

Journal of Field Robotics

View full text Add to dashboard Cite

Snake robots can be used in multiple tasks, like, climbing, surveillance, pipe inspection, welding, and so forth. Current gaits of snake robots do not support rotation on a single horizontal plane in a circular fashion of a pole or tree at a fixed height. This makes snake robot extremely difficult to take the end effector to the desired target position and orientation. In addition, torque requirement of individual actuators or links of snake robot proportionally increases based on the number of links to be lifted while performing any task. In this paper, we propose a new gait called circular gait using the Daisy Sequence Fitting algorithm to solve the problems of circular rotation on a horizontal plane at a certain height with low torque. A sliding mode (SM) controller is implemented to achieve the circular gait's required position dynamically. Simulation results show that using the proposed circular gait, the end effector can be moved to any point on the circumference of the fixed horizontal plane of the pole or tree with a lesser torque. For the 0.5 kg module, circular gait moved the end effector to the target point using only 4.5 N m torque. The SM controller outperforms the proportional, integral, and derivative controllers in terms of response characteristics.

show abstract

“…Considering the model-free feature of the reinforcement learning algorithm [28], combined with the offline training of the neural network, it can be directly used for the lateral control of autonomous cars and has a good compensation effect for the nonlinear characteristics of vehicles. Some recent work that uses Double Deep Q-learning Network •3• (DDQN) for path tracking shows promising results [29]. Furthermore, some researcher studies the possibility of combining MPC and deep reinforcement learning.…”

Section: Introductionmentioning

confidence: 99%

Modified Model Predictive Control based Adaptive Steering Strategy with Nonlinear Compensation by Using Double Deep Q-learning Network Algorithm

Lin

Chen

Zhou

et al. 2022

Preprint

View full text Add to dashboard Cite

Steering control for autonomous vehicles is used for more complex scenarios, such as nonlinear scenarios and varied vehicle speeds scenarios during the actual driving process. Model Predictive Control (MPC) is known as a feasible method for multi-constraints. However, a complicated mathematical model will lead to a great computational burden. To deal with this issue, a modified MPC-based adaptive steering strategy with nonlinear compensation by using Double Deep Q-learning Network Algorithm (DDQN) is proposed. Considering the real-time requirement for steering control, MPC is used as a basic controller to calculate the linear turning case. Then a DDQN algorithm is applied for compensating the errors caused by the nonlinear feature of the vehicle and the time-varying speed. Finally, numerical validations and experimental tests in a real vehicle are conducted to verify the effectiveness of the proposed control strategy. The traditional MPC and only the DDQN method are applied as the benchmark strategies. The comparison validation results under different speed and vehicle parameters indicate that the proposed method has superior performance in adapting to time-varying speeds and vehicle nonlinear features. And the experimental tests based on actual driving scenarios also validate the remarkable stability of the developed control method.

show abstract

Motion Planning for a Snake Robot using Double Deep Q-Learning

Cited by 12 publications

References 12 publications

Reinforcement learning-based motion control for snake robots in complex environments

Reinforcement learning-based motion control for snake robots in complex environments

Modeling of novel circular gait motion through daisy sequence fitting algorithm in a vertical climbing snake robot

Modified Model Predictive Control based Adaptive Steering Strategy with Nonlinear Compensation by Using Double Deep Q-learning Network Algorithm

Contact Info

Product

Resources

About