A novel model-free robust saturated reinforcement learning-based controller for quadrotors guaranteeing prescribed transient and steady state performance

Elhaki, Omid; Shojaei, Khoshnam

doi:10.1016/j.ast.2021.107128

Cited by 24 publications

(8 citation statements)

References 64 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In Reference 41, the actor‐NN is employed to estimate nonlinearities, actuator saturation nonlinearity and model uncertainties. The critic neural network is applied to estimate the reinforcement signals.…”

Section: Design Of Critic‐only Adp With Pgccmentioning

confidence: 99%

“…The online actor‐NN weight update law is created to help the actor‐NN perform the estimation policy more accurately online. Different from the role of neural networks in References 41 and 42, the critic‐only NN in this paper is used to approximate the proposed PHJB and solve the optimal controller instead of estimating parameters. Moreover, a significant low computational load for the controller can be acquired because of the freedom from system dynamics obtained by adopting actor‐critic neural networks model‐free control method in References 41 and 42.…”

Section: Design Of Critic‐only Adp With Pgccmentioning

confidence: 99%

“…Different from the role of neural networks in References 41 and 42, the critic‐only NN in this paper is used to approximate the proposed PHJB and solve the optimal controller instead of estimating parameters. Moreover, a significant low computational load for the controller can be acquired because of the freedom from system dynamics obtained by adopting actor‐critic neural networks model‐free control method in References 41 and 42. The linear PHJB is used to reduce the computational burden in this paper, which is carried out by deriving and transforming the conventional nonlinear HJB skillfully into the linear HJB through Equations (38), (39), and Theorem 2.…”

Section: Design Of Critic‐only Adp With Pgccmentioning

confidence: 99%

See 2 more Smart Citations

Parametrized guaranteed cost method with prescribed performance via critic‐only ADP for uncertain nonlinear systems and disturbances

Liu,

2024

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

In this paper, a novel parametrized guaranteed cost control (PGCC) method with prescribed performance control (PPC) is proposed for nonlinear systems with unknown bounded uncertainties and external disturbances described by Euler‐Lagrange (EL) equations. The proposed method is capable of ensuring a robust optimal performance in the presence of dynamics constraints without the specific information of nonlinear perturbation. As a consequence, the conditions in the form of disturbances and uncertainties are also relaxed. The augmented model is converted by transforming the uncertainties and disturbances in the original system to a mismatched term using asymmetric PPC. By designing and modifying the novel guaranteed cost function to account for the maximum value of the nonlinear perturbation, an infinite‐horizon PGCC problem is proposed by the unconstrained stationary optimal control problem. The relevant proposed linear parametrized Hamilton‐Jacobian‐Bellman (PHJB) equation is approximated to solve by a critic‐only neural network (NN) with efficient computationally. Uniformly ultimately bounded stability is guaranteed via a Lyapunov‐based stability analysis. Finally, numerical simulation results demonstrate the effectiveness of the proposed control scheme.

show abstract

Section: Design Of Critic‐only Adp With Pgccmentioning

confidence: 99%

Section: Design Of Critic‐only Adp With Pgccmentioning

confidence: 99%

Section: Design Of Critic‐only Adp With Pgccmentioning

confidence: 99%

See 1 more Smart Citation

Parametrized guaranteed cost method with prescribed performance via critic‐only ADP for uncertain nonlinear systems and disturbances

Liu,

2024

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

show abstract

“…Remark Compared with the RL‐based controller in [48], the proposed controller in this paper employs the concept of type‐3 fuzzy logic systems in the core of the designed RL method, which leads to a more professional learning, and uses a high‐gain observer to estimate the velocities and accelerations of the QUAV system in which the velocity/acceleration sensors are removed and weight, size and cost of the QUAV system is reduced. Hence, a more efficient closed‐loop control system is achieved.…”

Section: Output Feedback Rl‐based Controller Designmentioning

confidence: 99%

Robust state and output feedback prescribed performance interval type‐3 fuzzy reinforcement learning controller for an unmanned aerial vehicle with actuator saturation

Elhaki

Shojaei

Mohammadzadeh

2022

IET Control Theory & Appl

Self Cite

View full text Add to dashboard Cite

This paper presents a novel adaptive reinforcement learning control method with interval type-3 fuzzy neural networks to improve the trajectory tracking control performance of quadrotor unmanned aerial vehicles in challenging flight conditions. The proposed reinforcement learning controller is independent of the system's dynamics, and only relies on measurable signals of the system. An adaptive robust controller in collaboration with the suggested reinforcement learning method is designed to significantly improve the robustness of the control system. The maximum overshoot/undershoot, convergence rate and final tracking accuracy are ensured a priori by the prescribed performance control methodology. To develop the proposed controller and to achieve a high-performance closed-loop system, a high-gain observer is employed in order to estimate the velocity and acceleration of the quadrotor unmanned aerial vehicles system. The uniform ultimate boundedness stability of the proposed control algorithm is achieved by a Lyapunov-based stability analysis. Finally, in the simulation section, it is shown that the presented intelligent controller with the proposed learning algorithm result in a better performance in contrast to the other kind of conventional control techniques.

show abstract

“…Remarkably, more advanced RL algorithms such as soft actor-critic (SAC) [22], twin delayed deep deterministic policy gradient (TD3) [23] and proximal policy optimization (PPO) [24] are gradually being used in the control system of the quadrotor. An actor-critic neuralnetwork-based controller was presented in [25] to improve the quadrotor trajectory tracking performance. In [26], MAV has successfully completed autonomous navigation under a gust environment using the SAC algorithm as a DRL framework.…”

Section: Introductionmentioning

confidence: 99%

Improved Reinforcement Learning Using Stability Augmentation With Application to Quadrotor Attitude Control

Xue

et al. 2022

IEEE Access

View full text Add to dashboard Cite

Reinforcement learning (RL) has been successfully applied to motion control, without requiring accurate models and selection of control parameters. In this paper, we propose a novel RL algorithm based on proximal policy optimization algorithm with dimension-wise clipping (PPO-DWC) for attitude control of quadrotor. Firstly, dimension-wise clipping technique is introduced to solve the zerogradient problem of the PPO algorithm, which can quickly converge while maintaining good sampling efficiency, thus improving the control performance. Moreover, following the idea of stability augmentation system (SAS), a feedback controller is designed and integrated into the environment before training the PPO controller to avoid ineffective exploration and improve the system's convergence. The eventual controller consists of two parts: the first is the result of the actor neural network in the PPO algorithm, and the second is the output of the stability augmentation feedback controller. Both of them directly use an endto-end style of control commands to map the system state. This control architecture is applied in the attitude control of the quadrotor. The simulation results show that the quadrotor can quickly and accurately track the command and has a small steady-state error after the training by the improved PPO algorithm. Meanwhile, compared with the traditional PID controller and basic PPO algorithm, the proposed PPO-DWC algorithm with stability augmentation framework has better performance in tracking accuracy and robustness.

show abstract

A novel model-free robust saturated reinforcement learning-based controller for quadrotors guaranteeing prescribed transient and steady state performance

Cited by 24 publications

References 64 publications

Parametrized guaranteed cost method with prescribed performance via critic‐only ADP for uncertain nonlinear systems and disturbances

Parametrized guaranteed cost method with prescribed performance via critic‐only ADP for uncertain nonlinear systems and disturbances

Robust state and output feedback prescribed performance interval type‐3 fuzzy reinforcement learning controller for an unmanned aerial vehicle with actuator saturation

Improved Reinforcement Learning Using Stability Augmentation With Application to Quadrotor Attitude Control

Contact Info

Product

Resources

About