A partial policy iteration ADP algorithm for nonlinear neuro-optimal control with discounted total reward

Liang, Mingming; Wei, Qinglai

doi:10.1016/j.neucom.2020.11.014

Cited by 14 publications

(6 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Note that given a tuple of admissible control policies u j (j ∈ N), the value function V k i and the updated policies u k+1 i can be solved, simultaneously, without requiring the system dynamics. (27), and the same updated control policies as (28).…”

Section: Off-policy Irl Algorithmmentioning

confidence: 99%

“…Reinforcement learning (RL) is a biologically inspired approximation method, which can learn the optimal policy through continuous interaction with the environment 19‐24 . Inspired by the idea of RL, adaptive dynamic programming (ADP) has been proposed and used to solve the optimization problem of uncertain systems with single input effectively 25‐28 . In terms of optimal control, considering unknown disturbances and completely unknown system dynamics, Reference 29 develops an integral reinforcement learning method based on the actor‐critic structure.…”

Section: Introductionmentioning

confidence: 99%

“…[19][20][21][22][23][24] Inspired by the idea of RL, adaptive dynamic programming (ADP) has been proposed and used to solve the optimization problem of uncertain systems with single input effectively. [25][26][27][28] In terms of optimal control, considering unknown disturbances and completely unknown system dynamics, Reference 29 develops an integral reinforcement learning method based on the actor-critic structure. In Reference 30, an integral reinforcement learning method incorporating experience replay technique is presented to solve the optimal control problem of partially unknown constrained-input systems.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

Zhao

Chen

2022

Optim Control Appl Methods

View full text Add to dashboard Cite

This article studies the optimal tracking control problem of a class of multi‐input nonlinear system with unknown dynamics based on reinforcement learning (RL) and nonzero‐sum game theory. First of all, an augmented system composed of the tracking error dynamics and the command generator dynamics is constructed. Then, a tracking coupled Hamilton–Jacobi (HJ) equations associated with discounted cost function is derived, which gives the Nash equilibrium solution. The existence of Nash equilibrium is proved. To approximate the Nash equilibrium solution of tracking coupled HJ equations, we give two model‐based policy iteration (PI) algorithms, and analyze their equivalence and convergence. Further, to get rid of the prior knowledge of system dynamics, an off‐policy integral reinforcement learning (OP‐IRL) algorithm implemented by neural networks (NNs) is proposed. The weights of critic NNs and actor NNs are updated simultaneously by the gradient descent method. The convergence of the NNs weights and the stability of the closed‐loop error systems are proved. Finally, numerical simulation results are provided to demonstrate the effectiveness of the proposed OP‐IRL method.

show abstract

Section: Off-policy Irl Algorithmmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

Zhao

Chen

2022

Optim Control Appl Methods

View full text Add to dashboard Cite

show abstract

“…The adaptive dynamic programming method is suitable for solving the optimal control of complex nonlinear systems. But this method can only get an approximate optimal solution 2‐4 . The PID method is commonly used methods of ship motion control, and get good performance 5,6 .…”

Section: Introductionmentioning

confidence: 99%

“…But this method can only get an approximate optimal solution. [2][3][4] The PID method is commonly used methods of ship motion control, and get good performance. 5,6 But for the nonlinear of ship motion control, PID has poor robustness and it is easy to be saturation.…”

Section: Introductionmentioning

confidence: 99%

Multi‐verse optimizer based PI‐type active disturbance rejection generalized predictive control parameter tuning for ship motion

Ren¹,

Chen²,

Sun³

et al. 2021

Optim Control Appl Methods

View full text Add to dashboard Cite

The motion control of ships is the crucial of the transportation industry. The ship is an under‐actuated system with large inertia and nonlinearity, it also will be influenced by various interference during the voyaging. This work proposed a multi‐verse optimizer (MVO) tuned PI‐type active disturbance rejection generalized predictive control (PI‐ADRGPC) method. The PI‐ADRGPC is used to control the ship motion, and the MVO is employed to tune the parameters of PI‐ADRGPC. To verify the performance of the proposed method, a set of simulations with different interference are tested. The results indicate that compared with particle swarm optimization, dragonfly algorithm and moth‐flame optimization, the MVO tuned PI‐ADRGPC has better performance under different interference in the ship motion control. The Monte Carlo test also indicates that the robustness of the system is strong.

show abstract

Fully cooperative games with state and input constraints using reinforcement learning based on control barrier functions

Liu,

2023

Asian Journal of Control

View full text Add to dashboard Cite

This paper provides a novel safe reinforcement learning (RL) control algorithm to solve safe optimal problems for fully cooperative (FC) games of discrete‐time multiplayer nonlinear systems with state and input constraints. The FC game is a special case of nonzero‐sum (NZS) games, where all players cooperate to accomplish a common task. The algorithm is proposed based on the policy iteration (PI) framework utilizing only the measured data along the system trajectories in the environment. Different from most works about PI, an effective method of obtaining initial safe and stable control policies is given here. In addition, control barrier functions (CBFs) and an input constraint function are introduced to augment reward functions. And the monotonically nonincreasing property of the iterative value function in the PI algorithm maintains the safe set forward invariant. Then, the neural networks are employed to approximate the system dynamics, the iterative control policies, and the iterative value function, respectively. Furthermore, the proposed algorithm is supported by theoretical proofs that guarantee both safety and convergence. Finally, the effectiveness and safety of the algorithm are illustrated by the results of the simulation.

show abstract

A partial policy iteration ADP algorithm for nonlinear neuro-optimal control with discounted total reward

Cited by 14 publications

References 26 publications

Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

Off‐policy integral reinforcement learning‐based optimal tracking control for a class of nonzero‐sum game systems with unknown dynamics

Multi‐verse optimizer based PI‐type active disturbance rejection generalized predictive control parameter tuning for ship motion

Fully cooperative games with state and input constraints using reinforcement learning based on control barrier functions

Contact Info

Product

Resources

About