Integral reinforcement learning-based online adaptive event-triggered control for non-zero-sum games of partially unknown nonlinear systems

Su, Hanguang; Gong, Dunwei; Sun, Shaoxin; Cai, Yuliang

doi:10.1016/j.neucom.2019.09.088

Cited by 51 publications

(22 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that by utilizing the tuning law (29) for player i, an initial stabilizing control is generally needed. If the initial control is not stable, the stability of the system might not be ensured with the tuning law (29). In addition, a noise signal is usually requested to be added to the system to satisfy the persistence of excitation (PE) condition during the training process for the critic NN, which could influence the stability of the close-looped system.…”

Section: B Adaptive Controller Designmentioning

confidence: 99%

“…Lemma 1: Consider the NZS system (1). The critic tuning law and control policy for each player are given by (29) and (27), respectively. Let Assumptions 6-7 hold, thenW i is UUB for all i ∈ N.…”

Section: Stability Analysismentioning

confidence: 99%

“…A novel tuning law was adopted to increase the convergence speed of the controlled system. The integral reinforcement learning (IRL) technology, which is a crucial method of implementing ADP, was used to deal with the unknown dynamics for the NZS games [28], [29]. Recently, event-triggered mechanism has been widely employed to save transmission bandwidths and computing resources [30], [31].…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, a novel online method for multi-player non-zero-sum (NZS) differential games of nonlinear partially unknown continuous time (CT) systems with control constraints is developed based on neural networks (NN). The issue of multi-player NZS games with saturated actuator is elaborately analyzed and the unknown dynamics model is learned by applying identifier NN. Different from using the standard identifier-actor-critic framework of adaptive dynamic programming (ADP), the proposed method uses only identifier networks and critic networks for all the players to solve the coupled Hamilton-Jacobi (HJ) equations for multi-player NZS games, which could effectively simplify the algorithm and save computing resources. Moreover, a tuning law which utilizes the gradient descent method is designed for each critic network. Meanwhile, to remove the requirement for the initial stabilizing control, a novel stability term is designed to ensure the system stability during the training phase of the critic NN. By the means of Lyapunov approach, it is proven that the system states, the critic network weight estimation errors and the obtained control are all uniformly ultimately bounded (UUB). Finally, two numerical examples are simulated to illustrate the validity of the developed method for multi-player NZS games with control constraints. INDEX TERMS Adaptive critic designs, adaptive dynamic programming, control constraints, multi-player, non-zero-sum games.

show abstract

Section: B Adaptive Controller Designmentioning

confidence: 99%

Section: Stability Analysismentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Significant work has also emerged in the field of ADP research, such as optimal regulation, [36][37][38][39][40] tracking control, [41][42][43] and robust control, [44][45][46] etc. [47][48][49][50] However, there is less work to combine IRL with event-triggered control, and some existing works are on optimal regulation, 39 robust control, 45 and non-zero-sum games. 47 To the best of our knowledge, it is found that the work of IRL-based event-triggered constrained optimal tracking control has not been carried out yet.…”

Section: Introductionmentioning

confidence: 99%

“…[47][48][49][50] However, there is less work to combine IRL with event-triggered control, and some existing works are on optimal regulation, 39 robust control, 45 and non-zero-sum games. 47 To the best of our knowledge, it is found that the work of IRL-based event-triggered constrained optimal tracking control has not been carried out yet. Therefore, a novel algorithm that is not subjected to the initial admissible conditions for this problem is developed in this article, where both actor and critic use event-triggering schemes.…”

Section: Introductionmentioning

confidence: 99%

Adaptive dynamic programming‐based event‐triggered optimal tracking control

Xue

Luo

Liu

2021

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

In this article, an event-triggered constrained optimal tracking control algorithm using integral reinforcement learning (IRL) is developed. First, the constrained optimal tracking control problem is transformed into an optimal regulation problem by employing an augmented system with a discounted value function. Then, IRL is introduced to solve the Hamilton-Jacobi-Bellman equation, where the drift dynamics and reference dynamics are not required. The learning of neural network weights is event-triggered and there is no restriction on the initial control to be admissible. The involvement of event-triggering mechanism alleviates the pressure of data transmission on the network to some extent, which is suitable for control systems with limited computational and communication resources. Moreover, the nonexistence of Zeno behavior and the stability of the impulsive system are proved, respectively. Finally, the application of the algorithm on a mass-spring-damper system verifies its effectiveness.

show abstract

Event‐triggered‐based integral reinforcement learning output feedback optimal control for partially unknown constrained‐input nonlinear systems

Zou

Zhang

2023

Asian Journal of Control

View full text Add to dashboard Cite

In this paper, an adaptive output feedback event‐triggered optimal control algorithm is proposed for partially unknown constrained‐input continuous‐time nonlinear systems. First, a neural network observer is constructed to estimate unmeasurable state. Next, an event‐triggered condition is established, and only when the event‐triggered condition is violated will the event be triggered and the state be sampled. Then, an event‐triggered‐based synchronous integral reinforcement learning (ET‐SIRL) control algorithm with critic‐actor neural networks (NNs) architecture is proposed to solve the event‐triggered Hamilton–Jacobi–Bellman equation under the established event‐triggered condition. The critic and actor NNs are used to approximate cost function and optimal event‐triggered optimal control law, respectively. Meanwhile, the event‐triggered‐based closed‐loop system state and all the neural network weight estimation errors are uniformly ultimately bounded proved by Lyapunov stability theory, and there is no Zeno behavior. Finally, two numerical examples are presented to show the effectiveness of the proposed ET‐SIRL control algorithm.

show abstract

Integral reinforcement learning-based online adaptive event-triggered control for non-zero-sum games of partially unknown nonlinear systems

Cited by 51 publications

References 48 publications

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

Online Dual-Network-Based Adaptive Dynamic Programming for Solving Partially Unknown Multi-Player Non-Zero-Sum Games With Control Constraints

Adaptive dynamic programming‐based event‐triggered optimal tracking control

Event‐triggered‐based integral reinforcement learning output feedback optimal control for partially unknown constrained‐input nonlinear systems

Contact Info

Product

Resources

About