Reinforcement learning‐based tracking control for a quadrotor unmanned aerial vehicle under external disturbances

Abstract

This article addresses the high‐accuracy intelligent trajectory tracking control problem of a quadrotor unmanned aerial vehicle (UAV) subject to external disturbances. The tracking error systems are first reestablished by utilizing the feedforward control technique to compensate for the raw error dynamics of the quadrotor UAV. Then, two novel appointed‐fixed‐time observers are designed for the processed error systems to reconstruct the disturbance forces and torques, respectively. And the observation errors can converge to origin within the appointed time defined by users or designers. Subsequently, two novel control policies are developed utilizing reinforcement learning methodology, which can balance the control cost and control performance. Meanwhile, two critic neural networks are used to replace the traditional actor‐critic networks for approximating the solutions of Hamilton–Jacobi–Bellman equations. More specifically, two novel weight update laws are developed. They can not only update the weights of the critic neural networks online, but also avoid utilizing the persistent excitation condition innovatively. And that the ultimately uniformly bounded stability of the whole control system is proved according to Lyapunov method by utilizing the proposed reinforcement learning‐based control polices. Finally, simulation results are presented to illustrate the effectiveness and superior performances of the developed control scheme.

FTESO-adaptive neural network based safety control for a quadrotor UAV under multiple disturbances: algorithm and experiments

“…Liu et al (2021) used the strong approximation ability of radial basis function neural network (RBFNN) to estimate and compensate the uncertainty of model parameters. Liu et al (2023) used reinforcement learning technology to design control law, which solves the problem of high-precision tracking of UAV under external disturbances. However, the above methods have a large amount of computation and are difficult to implement on the hardware platform.…”

Fractional‐order integral terminal sliding‐mode control for perturbed nonlinear systems with application to quadrotors

“…The authors of Reference 38 developed a control strategy for autonomous quadrotor vehicle with external disturbances using SMC and backstepping approaches. In Reference 39, an intelligent control strategy based on reinforcement learning was developed for quadrotor vehicle.…”

An enhanced PI controller based on adaptive iterative learning control

“…Ma et al 6 proposed an improved dynamic linearization method to obtain an equivalent linear data model for learning systems. In recent years, learning‐based control methods, such as reinforcement learning, 7‐10 model predictive control, 11‐13 and iterative learning control (ILC), 14,15 have rapidly developed 16 . ILC aims to continuously improve control performance by using periodic system response data through iterative methods.…”

“…In recent years, learning-based control methods, such as reinforcement learning, [7][8][9][10] model predictive control, [11][12][13] and iterative learning control (ILC), 14,15 have rapidly developed. 16 ILC aims to continuously improve control performance by using periodic system response data through iterative methods. This method is widely used in many industrial scenarios, such as the robotics industry, 17 the semiconductor industry, 18 and chemical processes.…”