A novel neural network discrete‐time optimal control design for nonlinear time‐delay systems using adaptive critic designs

Abstract

In this article, a novel neural network (NN) optimal control approach using adaptive critic designs is developed for nonlinear discrete-time (DT) systems with time delays. First, to eliminate the delay term of control input, a time-delay matrix function is developed by designing a M network. Furthermore, the cost function is approximated by the critic NN, and the control signal can be obtained directly by using the information of critic NN according to the equilibrium condition. In addition, to shorten the learning time and reduce the computational burden in the control process, a novel control strategy with less adjustable parameters for the time-delay DT nonlinear systems is proposed in this article, in which the norm of the weight estimations of critic NN is updated to generate a novel long-term performance function.The proposed control algorithm using adaptive critic designs has the advantage of reducing adaptive learning parameters and lessening calculative burden. The Lyapunov stability analysis shows that the time-delay DT controlled systems can be uniformly ultimately bounded stable. Finally, three simulations are presented to demonstrate the control performance of the developed method. K E Y W O R D Sadaptive critic designs, discrete-time systems, neural network, optimal control, time delay 1 748

Observer-Based Finite-Time Inverse Optimal Output Regulation for Uncertain Nonlinear Systems

“…But for nonlinear systems, seeking the solutions of the Hamilton-Jacobi-Bellman (HJB) equation is an inevitable and difficult process when the optimal control problem is considered. In order to overcome this difficulty, neural networks were applied to approximate the solutions of the HJB equation, then the optimal control problem of nonlinear systems was addressed [29,30]. However, the approximate errors can be generated in this way, and the optimal control target will be affected if the errors are not small enough.…”

Adaptive critic optimization to decentralized event‐triggered control of continuous‐time nonlinear interconnected systems

“…As one of well‐known learning techniques, the iterative adaptive dynamic programming (ADP) algorithm has been widely used to solve a variety of robust optimal control problems, 1,2 tracking problems, 3‐6 optimal regulation problems, 7 multiagent differential game design, 8 and so on. Since it is difficult to solve the Hamilton–Jacobi–Bellman (HJB) equation directly, approximately solving the HJB equation is the main idea to address the optimal control problem based on ADP 9‐13 . In general, it is carried out by solving the HJB equation nearly and iteratively through a function approximation structure 14‐20 .…”

“…Since it is difficult to solve the Hamilton-Jacobi-Bellman (HJB) equation directly, approximately solving the HJB equation is the main idea to address the optimal control problem based on ADP. [9][10][11][12][13] In general, it is carried out by solving the HJB equation nearly and iteratively through a function approximation structure. [14][15][16][17][18][19][20] Therefore, the adaptive critic designs or adaptive critic structure-based optimization is the core idea of ADP.…”

Data‐driven optimal fault‐tolerant‐control and detection for a class of unknown nonlinear discrete‐time systems

“…For the cases of nonlinear discrete‐time systems and FTC approaches, they have a few works considering the optimal control problem such as RL, 19 H adaptive control, 2 ADP, 26 and action‐critic NNs 27 . To minimize the computation load, only critic NN has been developed in Reference 28 but the partial knowledge of the controlled plant such as the gain matrix has been required.…”