Suboptimal control for nonlinear slow‐fast coupled systems using reinforcement learning and Takagi–Sugeno fuzzy methods

Abstract

Summary In this article, by using singular perturbation theory, reinforcement learning (RL), and Takagi–Sugeno (T‐S) fuzzy methods, a RL‐fuzzy‐based composite suboptimal control method is proposed for nonlinear slow‐fast coupled systems (SFCSs) with unknown slow dynamics. First, the SFCSs is decomposed into slow and fast subsystems and the original optimal control problem is reduced to two subproblems. Then, for the slow subsystem, a nonlinear coordinate transformation is introduced to transform the nonquadratic slow utility function into the quadratic form. Unmeasurable virtual slow subsystem state is reconstructed by the state measurements of original system and slow controller design algorithm is proposed in the framework of RL by utilizing the actor‐critic neural networks to approximate the controller and cost function. For the fast subsystem, T‐S fuzzy model is established and state measurements of the original system are exploited to reconstruct the unmeasurable fast subsystem state. Fast controller is designed with the approach of parallel distributed compensation. The obtained slow and fast controllers form the composite suboptimal controller for the original SFCSs. Considering the state reconstruction error, convergence of the slow controller design algorithm, suboptimality of the composite controller, and stability of the closed‐loop SFCSs are analyzed. Finally, the effectiveness of our proposed method is illustrated by examples.

ADP-Based Optimal Control of Linear Singularly Perturbed Systems With Uncertain Dynamics: A Two-Stage Value Iteration Method

“…Since x xs and u xs are the state and input of the decomposed virtual subsystem (10), these signals cannot be measured from the full-order plant ( 8), (9). Thanks to Lemma 1, we replace x xs and u xs with x and u x during learning, which has also been commonly adopted in the past literature [2], [3], [10]- [16]. Then, we rewrite (27) as…”

“…The model matrices of the form (1), ( 2) are as follows [5], [11] A which are approximately identical to their optimal values. In addition, we list Table I to compare the performance between our method and existing methods [13]- [16].…”

“…But such a reduced feedback controller could cause the system to be far from its desired performance because of ignoring fast dynamics [5]. When the slow dynamics are unknown and the fast dynamics are known, some researchers posed ADP-based reduced-order composite control approaches for linear [13], interconnected [14], Markov jump [15], and nonlinear SPSs [16]. It should be pointed out, however, that these approaches are only available for standard SPSs, i.e., the internal dynamics of their fast subsystems are nonsingular, which are used to decompose system.…”

Design of a Takagi–Sugeno Fuzzy Exact Modeling of a Buck–Boost Converter

“…Takagi and Sugeno proposed a fuzzy model representing a linear relationship between the input and output of a nonlinear model by locally valid, interpolated linear dynamical systems [12][13][14]. Fuzzy TS is widely used for nonlinear systems; one example is in Liu et al [15]. The authors presented a suboptimal control with fuzzy TS methods and reinforcement learning (RL) for slow-fast coupled nonlinear systems.…”

Adaptive Neural Partial State Tracking Control for Full-State-Constrained Uncertain Singularly Perturbed Nonlinear Systems and Its Applications to Electric Circuit

“…Unfortunately, for SPSs with highly complex nonlinearities and strong couplings, it is hard to decompose them into slow-subsystems and fast-subsystems and thus it is generally not applicable to industrial systems. To overcome this issue, the methods on the basis of Takagi-Sugeno (T-S) fuzzy models [11][12][13][14][15][16] and neural networks (NNs) [17][18][19] have been suggested as an effective treatment. Using T-S fuzzy methods, the H ∞ control for fractionalorder SPSs in the presence of external disturbance and matched uncertainties based on sliding mode control method in [20].…”