A set‐based model‐free reinforcement learning design technique for nonlinear systems

Abstract

In this study, we propose an extremum-seeking approach for the approximation of optimal control problems for a class of unknown nonlinear dynamical systems. The technique combines a phasor extremum-seeking controller with a reinforcement learning strategy. The learning approach is used to estimate the value function of an optimal control problem of interest. The phasor extremum-seeking controller implements the approximate optimal controller. The approach is shown to provide reasonable approximations of optimal control problems without the need for a parameterization of the nonlinear system's dynamics. A simulation example is provided to demonstrate the effectiveness of the technique. KEYWORDS extremum-seeking control, phasor approximation, reinforcement learning Int J Adapt Control Signal Process. 2019;33:315-334.wileyonlinelibrary.com/journal/acs

Data-driven Outer-Loop Control Using Deep Reinforcement Learning for Trajectory Tracking

“…The incorporation of deep neural networks (DNN) as function approximators into the RL framework is known as deep reinforcement learning (DRL) . Complex control problems with discrete and continuous action spaces such as robotic manipulation, bipedal locomotion, and tracking control problems have been successfully solved using DRL [10,11,12,13,14]. Inspired by the ideas in RL, in this paper we explore the implementation of a non-invasive, full-online and model-free outer-loop control using DRL in the context of tracking control problems.…”

Editorial for the Special Issue on Learning‐based Adaptive Control: Theory and Applications

“…The model is assumed to be partially known, and an integral reinforcement learning approach is used to iteratively learn the solution of the associated algebraic Riccati equation online. The problem of optimal control over an infinite time horizon for an uncertain nonlinear system, affine in the control, is studied in the paper of Guay and Khalid . The idea proposed in this paper is based on the merger of reinforcement learning used to estimate the optimal cost function and an extremum seeker used to estimate the optimal control.…”