H_∞ optimal control of unknown linear systems by adaptive dynamic programming with applications to time‐delay systems

Abstract

In this paper, a new value iteration (VI) based method is proposed to solve the H ∞ control problem of continuous-time linear systems. The problem is transformed into solving a nonlinear differential equation, and the local stability of its solution is proved. Then a VI based iteration scheme for getting the approximation of the H ∞ optimal controller is proposed. Compared with the existing results, the proposed scheme does not need any special initial matrix or extra condition to solve the problem and provides a relatively small disturbance attenuation bound. The related result is also applied to the quadratic guaranteed cost control of linear time-delay systems. Simulation examples verify the effectiveness of the proposed schemes. K E Y W O R D Sadaptive dynamic programming, linear H ∞ control, model free control, time-delay systems, zero-sum differential games INTRODUCTIONOver the past few decades, the analysis and control of systems with disturbance and complex dynamics have received considerable attention in the control community. 1,2 As one of the most popular methods, H ∞ control, which studies the problem of the worst-case controller design, [3][4][5] was established and flourished. One way for solving the H ∞ control problem is to treat it as an infinite-horizon zero-sum game problem where the input of the system tries to minimize the infinite quadratic cost function and the disturbance tries to maximize it. Especially, by treating the delayed state as the disturbance, the H ∞ optimal control method can also solve the quadratic cost guaranteed control of systems with state delays. 6 The synthesis of the H ∞ controller needs to solve the Hamilton-Jacobi-Isaacs equations for nonlinear systems 7 and the game algebraic Riccati equation (GARE) for linear systems. 8 For linear systems, solving the GARE can be divided into two categories: the direct method 9,10 and the iterative method. 11,12 Both methods mentioned above need the exact system information of the original systems, which is however hard to be satisfied since in practice the system parameters usually contain uncertainties and thus are unknown. Therefore, how to solve the H ∞ optimal control problems without using the exact system parameters has attracted much attention in the literature.One way for dealing with the system uncertainties is to use the adaptive dynamic programming (ADP) method. In recent years, ADP has attracted more and more attention from both control theorists and engineers. It overcomes the curse of dimensionality 13,14 and acts as a promising methodology for solving the optimal control problem. [15][16][17][18] By ADP,

Modified general policy iteration based adaptive dynamic programming for unknown discrete‐time linear systems

“…▪ Proof of Lemma 5. Without loss of generality, we assume that P 0 > 0, since for any P 0 ≥ 0, by the iteration scheme (14) we always have P 1 > 0. Since {P i } is bounded, there exists P u > Q such that P j ≤ P u for j ∈ Z + .…”

“…By the fact that ‖P i * +1 − P i * ‖ < 𝜀, we have P i * +1 − P i * < 𝜀I, which indicates P i * +1 − P i * < Q. On the other hand, it follows from (14) that…”

“…This method is usually called the GPI method. As stated in Reference 8, the PI, VI, and GPI methods can be classified by the recursion number r in (14). The GPI can be seen as a general case of the PI method (r = ∞) and the VI method (r = 1).…”

“…Lemma 7 can also be seen as a standard GPI method for the optimal control problem with a discounted cost (described by the parameter γ). Also, when γ ≤ 1 is chosen large enough, the stability of the closed loop system with the (14).…”

“…In Reference 10 based on the VI method, four basic ADP implementation algorithms were introduced. Since then, the ADP method has been widely used to solve the optimal control problem for uncertain systems with continuous state and action spaces, for example, the constraint control problem, 11 the $$$ {H}_{\infty } $$$ control problem, 12‐15 the tracking problem, 16,17 the nonzero‐sum game problem, 18,19 the stochastic control problem, 20 and so on. In practical engineering area, the ADP technique was also successfully used to solve the fault‐tolerance control problem, 21 the optimal control of multi‐agent systems, 22 event‐triggered control problems, 23 and so on 24‐29 …”