Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm

Wei, Qinglai; Song, Ruizhuo; Li, Benkai; Lin, Xin

doi:10.1007/978-981-10-4080-1_5

Cited by 4 publications

(3 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To address this problem, approximation-based control strategies have been developed based on fuzzy logic system [19] and neural networks (NNs). NNs, with their expressive power of approximation widely recognized, have been utilized to represent direct dynamics, inverse dynamics, or a certain nonlinear mapping [20]- [22]. In general, NNsbased dynamics compensation can be classified into two main categories: either NN is used to produce control inputs for a feedback controller that is designed based on a nominal robot dynamic model, or NN directly estimates the dynamic system inversion.…”

Section: Introductionmentioning

confidence: 99%

Industrial Robot Trajectory Tracking Using Multi-Layer Neural Networks Trained by Iterative Learning Control

Chen¹,

Wen²

2019

Preprint

View full text Add to dashboard Cite

Section: Introductionmentioning

confidence: 99%

Industrial Robot Trajectory Tracking Using Multi-Layer Neural Networks Trained by Iterative Learning Control

Chen¹,

Wen²

2019

Preprint

View full text Add to dashboard Cite

“…For many traditional iterative ADP algorithms, it is required to build the model of nonlinear systems and then perform the ADP algorithms to derive an improved control policy [43][44][45][46][47][48][49][50]. These iterative ADP algorithms are denoted as "model-based ADP algorithms".…”

Section: Introductionmentioning

confidence: 99%

A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems

Wei

Liu

2015

Sci. China Inf. Sci.

Self Cite

View full text Add to dashboard Cite

In this paper, a novel iterative Q-learning algorithm, called "policy iteration based deterministic Qlearning algorithm", is developed to solve the optimal control problems for discrete-time deterministic nonlinear systems. The idea is to use an iterative adaptive dynamic programming (ADP) technique to construct the iterative control law which optimizes the iterative Q function. When the optimal Q function is obtained, the optimal control law can be achieved by directly minimizing the optimal Q function, where the mathematical model of the system is not necessary. Convergence property is analyzed to show that the iterative Q function is monotonically non-increasing and converges to the solution of the optimality equation. It is also proven that any of the iterative control laws is a stable control law. Neural networks are employed to implement the policy iteration based deterministic Q-learning algorithm, by approximating the iterative Q function and the iterative control law, respectively. Finally, two simulation examples are presented to illustrate the performance of the developed algorithm.

show abstract

“…In [11], interval type-2 fuzzy logic control and actor-critic RL algorithms with one-order digital low-pass filters are used to overcome the difficulties-caused by the nonlinearities and uncertainties of the system and the working environment-of accurate trajectory tracking control. An iterative adaptive dynamic programming (DP) technique is adopted to build the stable iterative Q-learning algorithm for achieving nonlinear neuro-optimal tracking control in [12]. To control a robotic manipulator with unknown parameters and dead zones, a neural network model for RL is described in [13].…”

mentioning

confidence: 99%

Reinforcement Learning Tracking Control for Robotic Manipulator With Kernel-Based Dynamic Model

Wang

Líu

et al. 2020

IEEE Trans. Neural Netw. Learning Syst.

View full text Add to dashboard Cite

Reinforcement learning (RL) is an efficient learning approach to solving control problems for a robot by interacting with the environment to acquire the optimal control policy. However, there are many challenges for RL to execute continuous control tasks. In this article, without the need to know and learn the dynamic model of a robotic manipulator, a kernelbased dynamic model for RL is proposed. In addition, a new tuple is formed through kernel function sampling to describe a robotic RL control problem. In this algorithm, a reward function is defined according to the features of tracking control in order to speed up the learning process, and then an RL tracking controller with a kernel-based transition dynamic model is proposed. Finally, a critic system is presented to evaluate the policy whether it is good or bad to the RL control tasks. The simulation results illustrate that the proposed method can fulfill the robotic tracking tasks effectively and achieve similar and even better tracking performance with much smaller inputs of force/torque compared with other learning algorithms, demonstrating the effectiveness and efficiency of the proposed RL algorithm. Index Terms-Kernel function, reinforcement learning (RL), reward function, robotics tracking control. I. INTRODUCTION I N RECENT years, the problem of tracking control for robotic manipulators has drawn considerable attention from both academia and industrial communities. To the matter of tracking control for a robotic manipulator, it is necessary to make each joint track a desired trajectory as close as Manuscript

show abstract

Nonlinear Neuro-Optimal Tracking Control via Stable Iterative Q-Learning Algorithm

Cited by 4 publications

References 43 publications

Industrial Robot Trajectory Tracking Using Multi-Layer Neural Networks Trained by Iterative Learning Control

Industrial Robot Trajectory Tracking Using Multi-Layer Neural Networks Trained by Iterative Learning Control

A novel policy iteration based deterministic Q-learning for discrete-time nonlinear systems

Reinforcement Learning Tracking Control for Robotic Manipulator With Kernel-Based Dynamic Model

Contact Info

Product

Resources

About