Online model-free controller for flexible wing aircraft: a policy iteration-based reinforcement learning approach

Abstract

The aerodynamic model of flexible wing aircraft is highly nonlinear with continuously time-varying dynamics under kinematic constraints. The nonlinearities stem from the aerodynamic forces and continuous deformations in the flexible wing. In spite of the various experimental attempts and theoretical setups that were made to model these dynamics, an accurate formulation was not achieved. The control paradigms of the aircraft are concerned with the electro-mechanical coupling between the pilot and the wing. It is challenging to design a flight controller for such aircraft while complying with these constraints. In this paper, innovative machine learning technique is employed to design a robust online model-free control scheme for flexible wing aircraft. The controller maintains internal asymptotic stability for the aircraft in real-time using selected set of measurements or states in uncertain dynamical environment. It intelligently incorporates the varying dynamics, geometric parameters, and physical constraints of the aircraft into optimal control strategies. The adaptive learning structure employs a policy iteration approach, taking advantage of Bellman optimality principles, to converge to an optimal control solution for the problem. Artificial neural networks are adopted to implement the adaptive learning algorithm in real-time without prior knowledge of the aerodynamic model of the aircraft. The control scheme is generalized and shown to function effectively for different pilot/wing control mechanisms. It also demonstrated its ability to overcome the undesired stability problems caused by coupling the pilot's dynamics with the flexible wing's frame of motion.

Iterative Learning Control for Load Frequency in Cyber-Attacked Multi-Area Power Systems

“…In this case, V is calculated using the modified Bellman equation proposed in [36]. The optimal Bellman equation, in this case, considers both the states of the model as well as the control input.…”

“…The main advantage of this algorithm is that the actorcritic neural network performs the approximation process in real time with no need for the dynamic model. As an extension of the actor-critic algorithm proposed in [36], [37], the DL model is proposed to estimate the system states that make the system reliable and more robust in the event of transmission delay and packet loss. In addition, the mathematical model discussed above is just used to generate a simulation and to take the measurement values.…”