Deep reinforcement learning based finite-horizon optimal tracking control for nonlinear system

Kim, Jong Woo; Park, Byung Jun; Yoo, Haeun; Lee, Jay H.; Lee, Jong Min

doi:10.1016/j.ifacol.2018.11.115

Cited by 11 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides, there are many more numerical methods to solve PDEs as in [49]- [52] with high processing requirements. Therefore, new methods based on machine learning (ML), e.g., (deep) reinforcement learning and neural-network-based online methods, are important to obtain the solution for PDEs with more accuracy or speed [10], [53]- [55]. The (deep) reinforcement learning methods are developed to learn the solution of the HJB equation and control rule in [53]- [55].…”

Section: Introductionmentioning

confidence: 99%

“…Therefore, new methods based on machine learning (ML), e.g., (deep) reinforcement learning and neural-network-based online methods, are important to obtain the solution for PDEs with more accuracy or speed [10], [53]- [55]. The (deep) reinforcement learning methods are developed to learn the solution of the HJB equation and control rule in [53]- [55]. In [10] a neural-network-based online solution of the HJB equation is used to explore the infinite horizon optimal robust guaranteed cost control of uncertain nonlinear systems.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Communication-Efficient Massive UAV Online Path Control: Federated Learning Meets Mean-Field Game Theory

Shiri

Park

Bennis

2020

IEEE Trans. Commun.

View full text Add to dashboard Cite

This paper investigates the control of a massive population of UAVs such as drones. The straightforward method of control of UAVs by considering the interactions among them to make a flock requires a huge inter-UAV communication which is impossible to implement in real-time applications. One method of control is to apply the mean field game (MFG) framework which substantially reduces communications among the UAVs. However, to realize this framework, powerful processors are required to obtain the control laws at different UAVs. This requirement limits the usage of the MFG framework for real-time applications such as massive UAV control. Thus, a function approximator based on neural networks (NN) is utilized to approximate the solutions of Hamilton-Jacobi-Bellman (HJB) and Fokker-Planck-Kolmogorov (FPK) equations. Nevertheless, using an approximate solution can violate the conditions for convergence of the MFG framework. Therefore, the federated learning (FL) approach which can share the model parameters of NNs at drones, is proposed with NN based MFG to satisfy the required conditions. The stability analysis of the NN based MFG approach is presented and the performance of the proposed FL-MFG is elaborated by the simulations.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Communication-Efficient Massive UAV Online Path Control: Federated Learning Meets Mean-Field Game Theory

Shiri

Park

Bennis

2020

IEEE Trans. Commun.

View full text Add to dashboard Cite

show abstract

“…Human-behavior learning [1] has become a popular research topic in different communities, specially in video games applications [2]. These kind of applications extract special features of the human performance while playing a video game using a deep reinforcement learning architecture known as Deep Q Network (DQN) [3]. The main components of this algorithm are: a deep neural network (designed with convolutional and fully connected layers [4]), a reinforcement learning update rule (Q-learning rule [5]- [7]), a εgreedy strategy for exploration-exploitation [8], [9] of the state-action space and an experience replay memory [10].…”

Section: Introductionmentioning

confidence: 99%

“…The main difference between these two approaches lies in the cognitions where different models and functions are used to extract experiences [12] and any previous knowledge [13], [14] that facilitates obtaining the solution of the desired task in an optimal way. Some examples of cognitive models are knowledge of the system and environment dynamics [9], [15] or any intelligent model/expert system [3], [16] like neural networks [17]- [19], function approximators [20]- [24], fuzzy systems [25], deep models [26], among others. The emotions defines a complex set and its topic for future work.…”

Section: Introductionmentioning

confidence: 99%

Human-Behavior Learning for Infinite-Horizon Optimal Tracking Problems of Robot Manipulators

Perrusquía

2021

2021 60th IEEE Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

In this paper, a human-behavior learning approach for optimal tracking control of robot manipulators is proposed. The approach is a generalization of the reinforcement learning control problem which merges the capabilities of different intelligent and control techniques in order to solve the tracking task. Three cognitive models are used: robot and reference dynamics and neural networks. The convergence of the algorithm is achieved under a persistent exciting and experience replay fulfillment. The algorithm learns online the optimal decision making controller according to the proposed cognitive models. Simulations were carry out to verify the approach using a 2-DOF planar robot.

show abstract

“…When system dynamics is unknown, the works 23‐26 establish neural networks as system identifiers to build the system dynamics through the collected data. Based on the system identifiers, the literature 23 uses ADP to solve the HJB equation where neural networks with time‐invariant weights are adopted to approximate the value function.…”

Section: Introductionmentioning

confidence: 99%

Two‐loop reinforcement learning algorithm for finite‐horizon optimal control of continuous‐time affine nonlinear systems

Chen

Xue

et al. 2021

Intl J Robust & Nonlinear

View full text Add to dashboard Cite

This article proposes three novel time‐varying policy iteration algorithms for finite‐horizon optimal control problem of continuous‐time affine nonlinear systems. We first propose a model‐based time‐varying policy iteration algorithm. The method considers time‐varying solutions to the Hamiltonian–Jacobi–Bellman equation for finite‐horizon optimal control. Based on this algorithm, value function approximation is applied to the Bellman equation by establishing neural networks with time‐varying weights. A novel update law for time‐varying weights is put forward based on the idea of iterative learning control, which obtains optimal solutions more efficiently compared to previous works. Considering that system models may be unknown in real applications, we propose a partially model‐free time‐varying policy iteration algorithm that applies integral reinforcement learning to acquiring the time‐varying value function. Moreover, analysis of convergence, stability, and optimality is provided for every algorithm. Finally, simulations for different cases are given to verify the convenience and effectiveness of the proposed algorithms.

show abstract

Deep reinforcement learning based finite-horizon optimal tracking control for nonlinear system

Cited by 11 publications

References 13 publications

Communication-Efficient Massive UAV Online Path Control: Federated Learning Meets Mean-Field Game Theory

Communication-Efficient Massive UAV Online Path Control: Federated Learning Meets Mean-Field Game Theory

Human-Behavior Learning for Infinite-Horizon Optimal Tracking Problems of Robot Manipulators

Two‐loop reinforcement learning algorithm for finite‐horizon optimal control of continuous‐time affine nonlinear systems

Contact Info

Product

Resources

About