Reinforcement-Learning-Based Robust Controller Design for Continuous-Time Uncertain Nonlinear Systems Subject to Input Constraints

Abstract

The design of stabilizing controller for uncertain nonlinear systems with control constraints is a challenging problem. The constrained-input coupled with the inability to identify accurately the uncertainties motivates the design of stabilizing controller based on reinforcement-learning (RL) methods. In this paper, a novel RL-based robust adaptive control algorithm is developed for a class of continuous-time uncertain nonlinear systems subject to input constraints. The robust control problem is converted to the constrained optimal control problem with appropriately selecting value functions for the nominal system. Distinct from typical action-critic dual networks employed in RL, only one critic neural network (NN) is constructed to derive the approximate optimal control. Meanwhile, unlike initial stabilizing control often indispensable in RL, there is no special requirement imposed on the initial control. By utilizing Lyapunov's direct method, the closed-loop optimal control system and the estimated weights of the critic NN are proved to be uniformly ultimately bounded. In addition, the derived approximate optimal control is verified to guarantee the uncertain nonlinear system to be stable in the sense of uniform ultimate boundedness. Two simulation examples are provided to illustrate the effectiveness and applicability of the present approach.

A Brief Review of Neural Networks Based Learning and Control and Their Applications for Robots

“…A policy iteration algorithm was introduced for infinite horizon optimal control of nonlinear systems using ADP in [67]. In [68], a reinforcement learning method was introduced for the stabilizing control of uncertain nonlinear…”

Neural Network Based Central Heating System Load Prediction and Constrained Control

“…It brings additional challenge for the control design. Some researches have been done for the input constraint problem [9][10][11]. In [12,13], an auxiliary system is novelly introduced to handle the constraint effect in a class of uncertain multiple-input multiple-output nonlinear system.…”

Data‐Driven Adaptive Critic Approach for Nonlinear Optimal Control via Least Squares Support Vector Machine

“…The general structure for implementing the ADP algorithm is actor-critic architecture, where the actor performs actions by interacting with its environment, and the critic evaluates actions and offers feedback information to the actor, leading to the improvement in performance of the subsequent actor [4]. Since then, various ADP methods have been developed [4][5][6][7][8][9][10][11][12][13][14]. However, most of the existing ADP-related results require an exact knowledge of nonlinear dynamics, which is often unavailable, thus these ADP algorithms are intractable to real-time control applications.…”