Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems

This paper develops an online adaptive critic algorithm based on policy iteration for partially unknown nonlinear optimal control with infinite horizon cost function. In the proposed method, only a critic network is established, which eliminates the action network, to simplify its architecture. The online least squares support vector machine (LS-SVM) is utilized to approximate the gradient of the associated cost function in the critic network by updating the input-output data. Additionally, a data buffer memory is added to alleviate computational load. Finally, the feasibility of the online learning algorithm is demonstrated in simulation on two example systems.

show abstract

“…[37]. The IRL-PI algorithm proposed converges uniformly to the optimal control solution on trajectories originating in Ω, i.e.…”

Section: (Policy Evaluation Step) Solve For the Value Functionmentioning

confidence: 97%

“…Consider the following affine in the control input nonlinear system with quadratic cost function applied in [37], which has the dynamics…”

Section: Nonlinear System Examplementioning

confidence: 99%

Data‐Driven Adaptive Critic Approach for Nonlinear Optimal Control via Least Squares Support Vector Machine

Sun

Liu

2017

Asian Journal of Control

View full text Add to dashboard Cite

show abstract

“…. , N. Remark 3: According to (42), it is obvious to observe that the approximate control policies of the N isolated subsystems can be derived directly based on the trained critic networks. Therefore, unlike the traditional actor-critic architecture, the action neural networks are not required any more.…”

Section: B Implementation Procedures Via Neural Networkmentioning

confidence: 99%

“…Abu-Khalaf and Lewis [41] derived an offline optimal control scheme for nonlinear systems with saturating actuators. Then, Vrabie and Lewis [42] and Vamvoudakis and Lewis [43] used online policy iteration algorithm to study the infinite horizon optimal control of continuous-time nonlinear systems, respectively. The former was performed based on the sequential updates of two neural networks, namely, critic network and action network, whereas in the latter, the two networks were trained simultaneously.…”

Section: Introductionmentioning

confidence: 99%

Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach

Liu

Wang

2014

IEEE Trans. Neural Netw. Learning Syst.

304

View full text Add to dashboard Cite

Abstract-In this paper, using a neural-network-based online learning optimal control approach, a novel decentralized control strategy is developed to stabilize a class of continuoustime nonlinear interconnected large-scale systems. First, optimal controllers of the isolated subsystems are designed with cost functions reflecting the bounds of interconnections. Then, it is proven that the decentralized control strategy of the overall system can be established by adding appropriate feedback gains to the optimal control policies of the isolated subsystems. Next, an online policy iteration algorithm is presented to solve the Hamilton-Jacobi-Bellman equations related to the optimal control problem. Through constructing a set of critic neural networks, the cost functions can be obtained approximately, followed by the control policies. Furthermore, the dynamics of the estimation errors of the critic networks are verified to be uniformly and ultimately bounded. Finally, a simulation example is provided to illustrate the effectiveness of the present decentralized control scheme.Index Terms-Adaptive dynamic programming, decentralized control, large-scale systems, neural networks, nonlinear interconnected systems, optimal control, policy iteration, reinforcement learning.

show abstract

“…In particular, biological experiments show that the dopamine neurotransmitter acts as a reinforcement signal which favors learning at the neuron level [12]. Based on reinforcement learning, a control signal can be generated for an agent to interact with unknown environments.…”

Section: Introductionmentioning

confidence: 99%

Reinforcement learning control for coordinated manipulation of multi-robots

Chen

Tee

et al. 2015

Neurocomputing

View full text Add to dashboard Cite

In this paper, coordination control is investigated for multi-robots to manipulate an object with a common desired trajectory. Both trajectory tracking and control input minimization are considered for each individual robot manipulator, such that possible disagreement between different manipulators can be handled. Reinforcement learning is employed to cope with the problem of unknown dynamics of both robots and the manipulated object. It is rigorously proven that the proposed method guarantees the coordination control of the multi-robots system under study. The validity of the proposed method is verified through simulation studies.

show abstract

Neural network approach to continuous-time direct adaptive optimal control for partially unknown nonlinear systems

Cited by 562 publications

References 20 publications

Data‐Driven Adaptive Critic Approach for Nonlinear Optimal Control via Least Squares Support Vector Machine

Data‐Driven Adaptive Critic Approach for Nonlinear Optimal Control via Least Squares Support Vector Machine

Decentralized Stabilization for a Class of Continuous-Time Nonlinear Interconnected Systems Using Online Learning Optimal Control Approach

Reinforcement learning control for coordinated manipulation of multi-robots

Contact Info

Product

Resources

About