Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

Abstract

This article studies the optimal tracking control problem of a class of multi‐input nonlinear system with unknown dynamics based on reinforcement learning (RL) and nonzero‐sum game theory. First of all, an augmented system composed of the tracking error dynamics and the command generator dynamics is constructed. Then, a tracking coupled Hamilton–Jacobi (HJ) equations associated with discounted cost function is derived, which gives the Nash equilibrium solution. The existence of Nash equilibrium is proved. To approximate the Nash equilibrium solution of tracking coupled HJ equations, we give two model‐based policy iteration (PI) algorithms, and analyze their equivalence and convergence. Further, to get rid of the prior knowledge of system dynamics, an off‐policy integral reinforcement learning (OP‐IRL) algorithm implemented by neural networks (NNs) is proposed. The weights of critic NNs and actor NNs are updated simultaneously by the gradient descent method. The convergence of the NNs weights and the stability of the closed‐loop error systems are proved. Finally, numerical simulation results are provided to demonstrate the effectiveness of the proposed OP‐IRL method.

Multi‐event‐triggered adaptive dynamic programming for non‐zero‐sum game of unknown nonlinear system

“…Meanwhile, the iterative methods of ADP include value iteration (VI) [16][17][18][19] and policy iteration (PI). [20][21][22] Ha et al 23 elaborated a new cost function to develop a VI-based ADP framework to solve the tracking control problem for unknown systems. In Reference 24, a data-driven iterative ADP was proposed to address the nonlinear optimal control problem.…”

“…Due to the widespread application of ADP, we can observe its extensions into areas such as NZSG, event‐triggered mechanism (ETM), and trajectory tracking. Meanwhile, the iterative methods of ADP include value iteration (VI) 16–19 and policy iteration (PI) 20–22 . Ha et al 23 elaborated a new cost function to develop a VI‐based ADP framework to solve the tracking control problem for unknown systems.…”

Event‐triggered neural experience replay learning for nonzero‐sum tracking games of unknown continuous‐time nonlinear systems

“…22 Thus, solving coupled Hamilton-Jacobi (HJ) equations is the key to deal with NZS game systems. RL method is applicable to continuous-time NZS game systems 23 and discrete-time NZS game systems. 24,25 The policy iteration (PI) of the ADP method is applied to yield the two-player control laws.…”