Nonlinear Adaptive Flight Control Using Incremental Approximate Dynamic Programming and Output Feedback

Abstract

A self-learning adaptive flight control for nonlinear systems allows a reliable, faulttolerant and effective operation of complex flight vehicles in a dynamic environment. Approximate dynamic programming provides a model-free control design for nonlinear systems with complex design processes and non-guaranteed closed-loop convergence properties. Linear approximate dynamic programming systematically applies a quadratic cost-togo function and greatly simplifies the design process of approximate dynamic programming. This paper presents a newly developed self-learning adaptive control method called incremental approximate dynamic programming for nonlinear unknown systems. It combines the advantages of linear approximate dynamic programming methods and incremental control techniques to generate a near-optimal control without a priori knowledge of the system model. In this paper, two incremental approximate dynamic programming algorithms with the direct availability of full states and with only the availability of system outputs have been developed. Both algorithms have been applied to a nonlinear aerospace related simulation model. The simulation results demonstrate that both model-free adaptive control algorithms improve the closed-loop performance of the nonlinear system, while keeping the design process simple and systematic as compared to conventional approximate dynamic programming algorithms.

Online Adaptive Incremental Reinforcement Learning Flight Control for a CS-25 Class Aircraft

“…Assuming a high sampling frequency and a slow-varying system, the incremental model as in Eq. 7, provides a linear, time-varying approximation of the nonlinear system [15,34]. An online system identification algorithm is utilized to generate estimates of the time-varying state and input matrix,…”

“…The offline identification phase itself requires a representative simulation model. In [29][30][31] novel frameworks, named Incremental Heuristic Dynamic Programming (IHDP) and Incremental Dual Heuristic Programming (IDHP) have been proposed to improve online adaptability and most importantly, eliminate the current need of an offline learning phase, by identifying an incremental model of the system in real-time. However, these novel frameworks have yet to be applied to and validated on complex, high-dimensional aerospace models and real systems.…”

Self-Healing Control for Large Launch Vehicle Based on Extended State Observer and Adaptive Dynamic Programming

“…In [29], an observer-critic structure-based ADP is proposed to handle the decentralized tracking control problem, and the Hamiltonian-Jacobi-Bellman equation is solved by a critic neural network. A model-free control scheme for a class of nonlinear system is devised in [30] based on an incremental approximate dynamic programming (I-ADP). Combing with ADP and sliding-mode control technique, the tracking control issue is solved in [31] for air-breathing hypersonic vehicles.…”

Adaptive Critic Control For Aircraft Lateral-Directional Dynamics

“…Until now several RL Flight Controllers have been proposed with different ACD frameworks. [12][13][14][15][16][17][18][19][20][21] One limitation of these controllers is that they have an exorbitant computational requirement. This requirement comes from learning two different functions with separate function approximation structures.…”