A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots

He, Naifeng; Yang, Zhong; Fan, Xiaoliang; Wu, Jiying; Sui, Yaoyu; Zhang, Qiuyan

doi:10.3390/act12080326

Cited by 3 publications

(2 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The controller output is determined by summing these three terms, with each multiplied by its respective tuning constant. The PID controller equation can be expressed mathematically as follows [42]:…”

Section: Pid Control Techniquementioning

confidence: 99%

Indirect Adaptive Control Using Neural Network and Discrete Extended Kalman Filter for Wheeled Mobile Robot

Silaa,

Bencherif,

Barambones

2024

Actuators

View full text Add to dashboard Cite

This paper presents a novel approach to address the challenges associated with the trajectory tracking control of wheeled mobile robots (WMRs). The proposed control approach is based on an indirect adaptive control PID using a neural network and discrete extended Kalman filter (IAPIDNN-DEKF). The proposed IAPIDNN-DEKF scheme uses the NN to identify the system Jacobian, which is used for tuning the PID gains using the stochastic gradient descent algorithm (SGD). The DEKF is proposed for state estimation (localization), and the NN adaptation improves the tracking error performance. By augmenting the state vector, the NN captures higher-order dynamics, enabling more accurate estimations, which improves trajectory tracking. Simulation studies in which a WMR is used in different scenarios are conducted to evaluate the effectiveness of the IAPIDNN-DEKF control. In order to demonstrate the effectiveness of the IAPIDNN-DEKF control, its performance is compared with direct adaptive NN (DA-NN) control, backstepping control (BSC) and an adaptive PID. On lemniscate, IAPIDNN-DEKF achieves RMSE values of 0.078769, 0.12086 and 0.1672. On sinusoidal trajectories, the method yields RMSE values of 0.01233, 0.015138 and 0.088707, and on sinusoidal with perturbation, RMSE values are 0.021495, 0.016504 and 0.090142 in x, y and θ, respectively. These results demonstrate the superior performance of IAPIDNN-DEKF for achieving accurate control and state estimation. The proposed IAPIDNN-DEKF offers advantages in terms of accurate estimation, adaptability to dynamic environments and computational efficiency. This research contributes to the advancement of robust control techniques for WMRs and showcases the potential of IAPIDNN-DEKF to enhance trajectory tracking and state estimation capabilities in real-world applications.

show abstract

Section: Pid Control Techniquementioning

confidence: 99%

Indirect Adaptive Control Using Neural Network and Discrete Extended Kalman Filter for Wheeled Mobile Robot

Silaa,

Bencherif,

Barambones

2024

Actuators

View full text Add to dashboard Cite

show abstract

“…At present, the flight control methods for drones mainly include linear control methods [8][9][10], nonlinear control methods [11][12][13], and intelligent control methods [14][15][16][17]. The learning-based robot control method has received widespread attention in the field of automatic control [18][19][20], as it ignores the dynamic model of the robot and learns control methods through a large amount of motion data. In the latest Nature journal, Elia Kaufmann et al [21], from the Robotics and Perception Group at the University of Zurich, studied the unmanned aerial vehicle autonomous system, Swift, which used deep reinforcement learning (DRL) algorithms to successfully defeat human champions in racing competitions, setting new competition records.…”

Section: Introductionmentioning

confidence: 99%

A Supervised Reinforcement Learning Algorithm for Controlling Drone Hovering

Wu,

Yang,

Zhuo

et al. 2024

Drones

Self Cite

View full text Add to dashboard Cite

The application of drones carrying different devices for aerial hovering operations is becoming increasingly widespread, but currently there is very little research relying on reinforcement learning methods for hovering control, and it has not been implemented on physical machines. Drone’s behavior space regarding hover control is continuous and large-scale, making it difficult for basic algorithms and value-based reinforcement learning (RL) algorithms to have good results. In response to this issue, this article applies a watcher-actor-critic (WAC) algorithm to the drone’s hover control, which can quickly lock the exploration direction and achieve high robustness of the drone’s hover control while improving learning efficiency and reducing learning costs. This article first utilizes the actor-critic algorithm based on behavioral value Q (QAC) and the deep deterministic policy gradient algorithm (DDPG) for drone hover control learning. Subsequently, an actor-critic algorithm with an added watcher is proposed, in which the watcher uses a PID controller with parameters provided by a neural network as the dynamic monitor, transforming the learning process into supervised learning. Finally, this article uses a classic reinforcement learning environment library, Gym, and a current mainstream reinforcement learning framework, PARL, for simulation, and deploys the algorithm to a practical environment. A multi-sensor fusion strategy-based autonomous localization method for unmanned aerial vehicles is used for practical exercises. The simulation and experimental results show that the training episodes of WAC are reduced by 20% compared to the DDPG and 55% compared to the QAC, and the proposed algorithm has a higher learning efficiency, faster convergence speed, and smoother hovering effect compared to the QAC and DDPG.

show abstract

An Overview of Model-Free Adaptive Control for the Wheeled Mobile Robot

Zhang,

Cen,

Huang

2024

WEVJ

View full text Add to dashboard Cite

Control technology for wheeled mobile robots is one of the core focuses in the current field of robotics research. Within this domain, model-free adaptive control (MFAC) methods, with their advanced data-driven strategies, have garnered widespread attention. The unique characteristic of these methods is their ability to operate without relying on prior model information of the control system, which showcases their exceptional capability in ensuring closed-loop system stability. This paper extensively details three dynamic linearization techniques of MFAC: compact form dynamic linearization, partial form dynamic linearization and full form dynamic linearization. These techniques lay a solid theoretical foundation for MFAC. Subsequently, the article delves into some advanced MFAC schemes, such as dynamic event-triggered MFAC and iterative learning MFAC. These schemes further enhance the efficiency and intelligence level of control systems. In the concluding section, the paper briefly discusses the future development potential and possible research directions of MFAC, aiming to offer references and insights for future innovations in control technology for wheeled mobile robots.

show abstract

A Self-Adaptive Double Q-Backstepping Trajectory Tracking Control Approach Based on Reinforcement Learning for Mobile Robots

Cited by 3 publications

References 38 publications

Indirect Adaptive Control Using Neural Network and Discrete Extended Kalman Filter for Wheeled Mobile Robot

Indirect Adaptive Control Using Neural Network and Discrete Extended Kalman Filter for Wheeled Mobile Robot

A Supervised Reinforcement Learning Algorithm for Controlling Drone Hovering

An Overview of Model-Free Adaptive Control for the Wheeled Mobile Robot

Contact Info

Product

Resources

About