H ∞ tracking control for linear discrete‐time systems via reinforcement learning

In this paper, a new value iteration (VI) based method is proposed to solve the H ∞ control problem of continuous-time linear systems. The problem is transformed into solving a nonlinear differential equation, and the local stability of its solution is proved. Then a VI based iteration scheme for getting the approximation of the H ∞ optimal controller is proposed. Compared with the existing results, the proposed scheme does not need any special initial matrix or extra condition to solve the problem and provides a relatively small disturbance attenuation bound. The related result is also applied to the quadratic guaranteed cost control of linear time-delay systems. Simulation examples verify the effectiveness of the proposed schemes. K E Y W O R D Sadaptive dynamic programming, linear H ∞ control, model free control, time-delay systems, zero-sum differential games INTRODUCTIONOver the past few decades, the analysis and control of systems with disturbance and complex dynamics have received considerable attention in the control community. 1,2 As one of the most popular methods, H ∞ control, which studies the problem of the worst-case controller design, [3][4][5] was established and flourished. One way for solving the H ∞ control problem is to treat it as an infinite-horizon zero-sum game problem where the input of the system tries to minimize the infinite quadratic cost function and the disturbance tries to maximize it. Especially, by treating the delayed state as the disturbance, the H ∞ optimal control method can also solve the quadratic cost guaranteed control of systems with state delays. 6 The synthesis of the H ∞ controller needs to solve the Hamilton-Jacobi-Isaacs equations for nonlinear systems 7 and the game algebraic Riccati equation (GARE) for linear systems. 8 For linear systems, solving the GARE can be divided into two categories: the direct method 9,10 and the iterative method. 11,12 Both methods mentioned above need the exact system information of the original systems, which is however hard to be satisfied since in practice the system parameters usually contain uncertainties and thus are unknown. Therefore, how to solve the H ∞ optimal control problems without using the exact system parameters has attracted much attention in the literature.One way for dealing with the system uncertainties is to use the adaptive dynamic programming (ADP) method. In recent years, ADP has attracted more and more attention from both control theorists and engineers. It overcomes the curse of dimensionality 13,14 and acts as a promising methodology for solving the optimal control problem. [15][16][17][18] By ADP,

“…where Q , R and S are all given positive definite matrices. Moreover, if u(t) is designed as (23), then the cost function defined in (22) will be bounded by…”

Section: Application To Unknown Linear Time-delay Systemsmentioning

confidence: 99%

mentioning

confidence: 99%

H_∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems

Jiang

Zhou

Liu

2021

“…Bhattacharya et al 12 worked on a visibility‐based PE game when the environment contains a circular obstacle. Furthermore, Li et al 13 and Liu et al 14 developed an reinforcement learning (RL) algorithm to learn the Nash equilibrium solution for designing model‐free controller by solving the game algebraic Riccati equation forward in time.…”

Section: Introductionmentioning

confidence: 99%

Optimal game theoretic solution of the pursuit‐evasion intercept problem using on‐policy reinforcement learning

Kartal

Subbarao

Dogan

et al. 2021

This article presents a rigorous formulation for the pursuit‐evasion (PE) game when velocity constraints are imposed on agents of the game or players. The game is formulated as an infinite‐horizon problem using a non‐quadratic functional, then sufficient conditions are derived to prove capture in a finite‐time. A novel tracking Hamilton–Jacobi–Isaacs (HJI) equation associated with the non‐quadratic value function is employed, which is solved for Nash equilibrium velocity policies for each agent with arbitrary nonlinear dynamics. In contrast to the existing remedies for proof of capture in PE game, the proposed method does not assume players are moving with their maximum velocities and considers the velocity constraints a priori. Attaining the optimal actions requires the solution of HJI equations online and in real‐time. We overcome this problem by presenting the on‐policy iteration of integral reinforcement learning (IRL) technique. The persistence of excitation for IRL to work is satisfied inherently until capture occurs, at which time the game ends. Furthermore, a nonlinear backstepping control method is proposed to track desired optimal velocity trajectories for players with generalized Newtonian dynamics. Simulation results are provided to show the validity of the proposed methods.

“…For example, adaptive dynamic programming algorithm is an effective DDC method to deal with time-varying trajectory tracking problem. [33][34][35][36] However, an assumption y d (k + 1) = Fy d (k) is usually provided for the desired time-varying trajectory to construct the augmented system, where F is a constant matrix. This assumption is not required for the indirect data-driven method 37 and MFAC.…”

Section: Introductionmentioning

confidence: 99%

Model‐free adaptive tracking control for networked nonlinear systems with data dropout

Wang

2021

Self Cite

The data-driven tracking control problem is investigated for networked nonlinear systems subjected to data dropout. The desired time-varying output trajectory is considered in this article, which is more general than the constant trajectory. In order to improve the tracking performance, the change rate of tracking error is additionally introduced to the performance index to design model-free adaptive controller. Accordingly, more adjustable parameters are introduced into the controller. With the help of the approximation ability of RBFNN, an RBFNN estimation algorithm is designed to compensate for the adverse effects of data dropout. It is shown that the proposed controller can guarantee the system output to track the desired time-varying output trajectory.Finally, two numerical simulations are provided to illustrate the effectiveness of the proposed approach.