Optimal tracking control for non‐zero‐sum games of linear discrete‐time systems via off‐policy reinforcement learning

Abstract

SummaryIn this article, a model‐free off‐policy reinforcement learning algorithm is applied to address the optimal tracking problem based on multiplayer non‐zero‐sum games for discrete‐time linear systems. In contrast to the traditional method and the policy iteration method for solving the optimal tracking problems, the proposed algorithm operates with the system data rather than the knowledge of the system dynamics. For performing the proposed algorithm, an auxiliary augmented system is constructed via assembling the original system and the reference trajectory while a discount factor is introduced into the performance indexes. It is analyzed that the solutions of the proposed algorithm converge to the Nash equilibrium and the result is not influenced by the probing noise. Two simulations are presented to verify the feasibility and effectiveness of the proposed algorithm.

Data‐driven optimal fault‐tolerant‐control and detection for a class of unknown nonlinear discrete‐time systems

“…The initial setting of PPD is given as θ(1) = 1. All parameters in the learning law (19) and ( 20 (20). 8 For the case of unknown system dynamics, those parameters m and b are designed by trial-and-error.…”

“…For the case of unknown system dynamics, controllers based on reinforcement learning (RL) have been developed with the structure action‐critic NNs 18,19 . The nearly optimal solution can be obtained under the long‐term performance of Bellman equation 20 . Because of the future value in the Bellman equation, the critic NNs have been tuned by temporal difference (TD) which can cause the learning performance for the TD error for estimating the solution of Hamilton–Jacobi–Bellman (HJB) 21,22 .…”

“…18,19 The nearly optimal solution can be obtained under the long-term performance of Bellman equation. 20 Because of the future value in the Bellman equation, the critic NNs have been tuned by temporal difference (TD) which can cause the learning performance for the TD error for estimating the solution of Hamilton-Jacobi-Bellman (HJB). 21,22 For unknown system dynamics, a data-driven scheme has been successfully proposed in the work 23 but it has limitations for a class of linear continuous-time only.…”

Optimization of electroosmotic flow to enhance the removal of contaminants from low‑permeable soils

“…However, the removal rate of ionic charge species in soil can be influenced by electrical conductivity, mineral composition, porosity, and tortuosity as well as the physiochemical properties of soil [10]. The significant drawbacks of EK are the electrode corrosion, dehydration, thermal effects, and focusing effects (which occur in soil, where acidic (H + ions) and alkaline (OHˉ ions) fronts collide with each other and cause the accumulation of metal ions in the soil and restrict their emissions) [11]. In particular, the focusing effects are the main thing that makes it hard to remove heavy metals over a long period of time [12].…”

Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

“…As we all know, RL methods have been widely used to solve the optimal control of NZS games, but few results considered solving the optimal tracking control problem of NZS games. A model‐free off‐policy RL algorithm is proposed to solve the optimal tracking problem of discrete‐time linear NZS games in Reference 51. Using RL methods, the output regulation linear optimal tracking control NZS game problem in discrete‐time and continuous‐time are studied in References 52 and 53, respectively.…”