Heuristic Dynamic Programming Algorithm for Optimal Control Design of Linear Continuous-Time Hyperbolic PDE Systems

Abstract

This work considers the optimal control problem of linear continuous-time hyperbolic partial differential equation (PDE) systems with partially unknown system dynamics. To respect the infinite-dimensional nature of the hyperbolic PDE system, the problem can be reduced to finding a solution of the space-dependent Riccati differential equation (SDRDE), which requires the full system model. Therefore, a heuristic dynamic programming (HDP) algorithm is proposed to achieve online optimal control of the hyperbolic PDE system, which online collects data accrued along system trajectories and learns the solution of the SDRDE without requiring the internal system dynamics. The convergence of HDP algorithm is established by showing that the HDP algorithm generates a nondecreasing sequence which uniformly converges to the solution of the SDRDE. For implementation purposes, the HDP algorithm is realized by developing an approximate approach based on the method of weighted residuals. Finally, the application on a steam-jacketed tubular heat exchanger demonstrates the effectiveness of the developed control approach.

Non-fragile guaranteed cost fuzzy control for nonlinear first-order hyperbolic partial differential equation systems

“…Remark 1 It should be pointed out that hyperbolic PDE systems represent the dynamics of quantities of industrial processes, such as plug-flow reactors, fixed-bed reactors, tubular heat exchangers, Saint-Venant equations for open channel hydraulic systems, traffic flows, and wave equations, and so on [19,25,27,31]. Some special cases of nonlinear first-order hyperbolic PDE system (1) have been studied in [2,7,16,19,31], where the systems are linear or quasi-linear hyperbolic PDE systems.…”

“…Some special cases of nonlinear first-order hyperbolic PDE system (1) have been studied in [2,7,16,19,31], where the systems are linear or quasi-linear hyperbolic PDE systems. Up to now, few fuzzy control design approaches for nonlinear firstorder hyperbolic PDE systems are available except [25,27,32].…”

“…Chen et al modes. This suggests one should start form the infinite-dimensional model itself to control the nonlinear hyperbolic PDE systems, and some control approaches have been reported, including the optimal control method [1,2], the sliding mode control method [10,20], the model predictive control method [19], the nonlinear control method through a combination of PDE theory and geometric control techniques [7] and the heuristic dynamic programming (HDP) method [31].Over the past few decades, there has been rapidly growing interest in fuzzy control of nonlinear systems. Particularly, since the so-called Takagi-Sugeno (T-S) fuzzy model [21] was proposed, as a popular and powerful tool in approximating a complex nonlinear system, it has been made a great of results [3,5,6,9,11,[22][23][24].…”

“…Chen et al modes. This suggests one should start form the infinite-dimensional model itself to control the nonlinear hyperbolic PDE systems, and some control approaches have been reported, including the optimal control method [1,2], the sliding mode control method [10,20], the model predictive control method [19], the nonlinear control method through a combination of PDE theory and geometric control techniques [7] and the heuristic dynamic programming (HDP) method [31].…”

Off-Policy Reinforcement Learning for <inline-formula> <tex-math notation="LaTeX">$ H_\infty $ </tex-math></inline-formula> Control Design

“…In [19], an online neural network (NN) based decentralized control strategy was developed for stabilizing a class of continuous-time nonlinear interconnected large-scale systems. In addition, it worth mentioning that the thought of RL methods have also been introduced to solve the optimal control problem of partial differential equation systems [6], [12], [15], [16], [23]. However, for most of practical real systems, the existence of external disturbances is usually unavoidable.…”