Robust tracking control with reinforcement learning for nonlinear‐constrained systems

Abstract

This article considers the robust tracking control problem of uncertain nonlinear systems with asymmetric input constraints. Initially, the tracking error dynamics and the desired trajectory dynamics are constructed as an augmented system. Then, with a discounted value function being introduced for the nominal augmented system, the original tracking control problem is transformed into a constrained optimal control problem. To solve the constrained optimal control problem, its related Hamilton–Jacobi–Bellman equation (HJBE) is developed. After that, a critic approximator is constructed to solve the HJBE in the reinforcement learning framework. To tune the parameters used in the critic approximator, a novel concurrent learning technique is introduced, which could remove the persistence of excitation condition. Moreover, the uniform ultimate boundedness of the tracking error and the parameters' estimation error of the critic approximator is assured via the Lyapunov's approach. Finally, a spring‐mass‐damper mechanical system and a robot manipulator system are given to validate the theoretical claims.

Reinforcement Learning-Based Decentralized Safety Control for Constrained Interconnected Nonlinear Safety-Critical Systems

“…Due to accounting for safety constraints and asymmetric input constraints in ( 7 ), the optimal control law does not converge to zero while the system state achieves the stable phase [ 43 ]. The discount factor , may be unbounded, so it is necessary to consider the discount factor.…”

Optimal control of partially unknown constrained‐input systems: A dynamic event‐triggered‐based approach

“…3 However, the HJB equation of nonlinear systems is a nonlinear partial differential equation, and its analytical solution is difficult or even impossible to obtain. [4][5][6] Therefore, how to solve the HJB equation becomes the first challenge to be faced. Adaptive dynamic programming (ADP) using neural network (NN) approximation techniques is a powerful method for solving the HJB equation in optimization problems, [7][8][9] and many ADP-based methods [10][11][12] have been proposed.…”

“…Solving the optimal control problem is mainly to solve the Hamilton–Jacobi–Bellman (HJB) equation 3 . However, the HJB equation of nonlinear systems is a nonlinear partial differential equation, and its analytical solution is difficult or even impossible to obtain 4‐6 . Therefore, how to solve the HJB equation becomes the first challenge to be faced.…”

Robust dynamic event‐triggered control of saturated nonlinear systems using reinforcement learning

“…If ignoring saturation during the design of controllers, the systems' performance often becomes awful and unacceptable. 14 Abu-Khalaf and Lewis 15 first introduced an off-line RL approach to solve the optimal control problem of continuous-time saturated nonlinear systems. After that, Luo et al 16 suggested an off-policy RL algorithm to derive the optimal control of discrete-time saturated nonlinear systems.…”

“…Many actuators are often subject to saturation because of physical restrictions, such as temperature and pressure. If ignoring saturation during the design of controllers, the systems' performance often becomes awful and unacceptable 14 . Abu‐Khalaf and Lewis 15 first introduced an off‐line RL approach to solve the optimal control problem of continuous‐time saturated nonlinear systems.…”