Deterministic policy gradient adaptive dynamic programming for model-free optimal control

Zhang, Yongwei; Zhao, Bo; Liu, Derong

doi:10.1016/j.neucom.2019.11.032

Cited by 49 publications

(13 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this paper, the system functions f i (x i ) and g i (x i ) are both considered to be known, which may limit its application in practice. Inspired by the recent works of Lin et al 53 and Zhang et al, 54 our future work aims at extending the ETDOFTC method to nonlinear systems with unknown dynamics.…”

Section: Discussionmentioning

confidence: 99%

Event‐triggered decentralized optimal fault tolerant control for mismatched interconnected nonlinear systems through adaptive dynamic programming

Luo

Zhao

Liu

2021

Optim Control Appl Methods

Self Cite

View full text Add to dashboard Cite

In this paper, we propose an event-triggered decentralized optimal fault tolerant control (ETDOFTC) scheme based on adaptive dynamic programming for mismatched interconnected nonlinear systems with actuator failures. For fault-free dynamic models, the decentralized control problem is addressed by developing a set of decentralized optimal control strategies for isolated subsystems with modified value functions, which are approximated by critic neural networks. Meanwhile, the neural network-based decentralized observer is established to approximate actuator failures and mismatched unknown interconnections.The weights of the neural networks are aperiodically updated at the designed triggering instants. Then, the proposed ETDOFTC scheme is obtained by combining the event-triggered decentralized optimal control strategies with the adaptive fault compensators. Furthermore, it is proved that all the signals of the closed-loop system are uniformly ultimately bounded via the Lyapunov stability analysis. Finally, simulation results of two examples are presented to confirm the effectiveness of the proposed ETDOFTC scheme. K E Y W O R D Sadaptive dynamic programming, decentralized control, event-triggered control, fault tolerant control, mismatched interconnected nonlinear systems, neural networks, reinforcement learning

show abstract

Section: Discussionmentioning

confidence: 99%

Event‐triggered decentralized optimal fault tolerant control for mismatched interconnected nonlinear systems through adaptive dynamic programming

Luo

Zhao

Liu

2021

Optim Control Appl Methods

Self Cite

View full text Add to dashboard Cite

show abstract

“…A torsional pendulum system, 34 which is an affine nonlinear system, is used in this simulation to evaluate the effectiveness of the proposed CBFADP algorithm. The dynamics are given by: where M = 1 3 kg and l = 2 3 m denote mass and the length of the pendulum bar, respectively.…”

Section: Nonlinear Systemmentioning

confidence: 99%

Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

Wang

Rao

et al. 2021

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

Adaptive dynamic programming (ADP) methods have demonstrated their efficiency. However, many of the applications for which ADP offers great potential, are also safety-critical and need to meet safety specifications in the presence of physical constraints. In this article, an optimal controller for solving discrete-time nonlinear systems with state constraints is proposed. By introducing the control barrier function into the utility function, the problem with state constraints is transformed into an unconstrained optimal control problem, addressing state constraints which are difficult to handle by traditional ADP methods. The constructed sequence of value function is shown to be monotonically non-increasing and converges to the optimal value. Besides, this article gives the stability proof of the developed algorithm, as well as the conditions for satisfying the state constraints. To implement and approximate the control barrier function based adaptive dynamic programming algorithm, an actor-critic network structure is built. During the training process, two neural networks are used for approximation separately. The performance of the proposed method is validated by testing it on a simulation example.

show abstract

“…Before the main theorem, the following assumption which is common in ADP literatures 39‐43 is provided.…”

Section: Adaptive Critic Designs Based Event‐triggered Tracking Control Schemementioning

confidence: 99%

Event‐triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs

Zhang

Zhao

Liu

et al. 2021

Intl J Robust & Nonlinear

Self Cite

View full text Add to dashboard Cite

In this article, the event‐triggered optimal tracking control problem for multiplayer unknown nonlinear systems is investigated by using adaptive critic designs. By constructing a neural network (NN)‐based observer with input–output data, the system dynamics of multiplayer unknown nonlinear systems is obtained. Subsequently, the optimal tracking control problem is converted to an optimal regulation problem by establishing a tracking error system. Then, the optimal tracking control policy for each player is derived by solving coupled event‐triggered Hamilton‐Jacobi (HJ) equation via a critic NN. Meanwhile, a novel weight updating rule is designed by adopting concurrent learning method to relax the persistence of excitation (PE) condition. Moreover, an event‐triggering condition is designed by using Lyapunov's direct method to guarantee the uniform ultimate boundedness (UUB) of the closed‐loop multiplayer systems. Finally, the effectiveness of the developed method is verified by two different multiplayer nonlinear systems.

show abstract

Deterministic policy gradient adaptive dynamic programming for model-free optimal control

Cited by 49 publications

References 29 publications

Event‐triggered decentralized optimal fault tolerant control for mismatched interconnected nonlinear systems through adaptive dynamic programming

Event‐triggered decentralized optimal fault tolerant control for mismatched interconnected nonlinear systems through adaptive dynamic programming

Adaptive dynamic programming for optimal control of discrete‐time nonlinear system with state constraints based on control barrier function

Event‐triggered optimal tracking control of multiplayer unknown nonlinear systems via adaptive critic designs

Contact Info

Product

Resources

About