An algorithm to solve the discrete HJI equation arising in the L2 gain optimization problem

Zero-sum two-player game theoretic formulation of affine nonlinear discrete-time systems using neural networks

“…Although in [8][9], a Riccati equation based solution to the HJI equation was found (without using NN) nicely using Taylor series expansion of the system dynamics as well as the value function, the optimal policy, and worse case disturbance are obtained in a fundamentally different manner since we do not require Taylor series expansions of the system dynamics, optimal policy, or worse case disturbance while only the Taylor series expansion of value function under small perturbation assumption is needed. Additionally, the sufficient conditions for existence of saddle-point and the minimum disturbance gain (nonlinear H 00 optimal control problem) are rigorously shown in this work as opposed to [8] [9].…”

“…Similarly, [8] [9] requires the knowledge of the system dynamics and uses Taylor series expansion of the system dynamics.…”

“…To overcome this problem, the continuous and discrete time problems are addressed in [7] and [8] [9], respectively, where a solution is found by using the Taylor series expansion and solving the coefficients using Riccati equations. Thus, the HJI problem is reduced to solving a Riccati equation and a sequence of linear algebraic equations.…”

“…While the continuous-time HJI problem has been under consideration [7] [10][11], discrete-time robust optimal control problem for nonlinear systems is addressed in a limited manner [8] [9].…”

Heuristic Dynamic Programming Nonlinear Optimal Controller

“…The discrete-time nonlinear optimal control solution relies on solving the discrete-time (DT) Hamilton-Jacobi-Bellman (HJB) equation (Lewis & Syrmos, 1995), exact solution of which is generally impossible for nonlinear systems. Solutions to the DT HJB equation with known dynamics and continuous state space and action space were given in (Huang, 1999), where the coefficients of the Taylor series expansion of the value function are systematically computed. In (Chen & Jagannathan, 2005), the authors show that under certain conditions a second-order approximation of the discrete-time (DT) Hamilton-Jacobi-Bellman (HJB) equation can be considered; under those conditions discussed in that paper, the authors solve for the value function that satisfies the second order expansion of the DT HJB instead of solving for the original DT HJB.…”