Reinforcement learning‐based adaptive optimal tracking algorithm for Markov jump systems with partial unknown dynamics

Abstract

In this article, a novel method is proposed to solve the adaptive optimal tracking algorithm for a class of Markov jump systems. First, the augmented system with the tracking signal is built under the decoupling Markov jump systems and it is proved that the selected performance index satisfies the algebraic Riccati equation which can be solved by policy iteration schemes. Then, a reinforcement learning (RL) algorithm is used to solve the coupled algebraic Riccati equations by using partial knowledge of system dynamics. The convergence of the partial model-free integral RL iteration algorithm is also proved. Finally, a simulation example is given to show the better tracking effectiveness and accuracy of the online iteration algorithm comparing with the offline one.

A Fuzzy-Model-Based Approach to Optimal Control for Nonlinear Markov Jump Singularly Perturbed Systems: A Novel Integral Reinforcement Learning Scheme

“…Based on the research in [18], work [19] settled the optimal control problem of completely unknown MJSs in view of the off-policy algorithm. Along with this line, in [20], Tu et al used a parallel control algorithm to solve the tracking control problem for MJSs. Besides, thorough seeking the Nash equilibrium for zero-sum game, the H ∞ control problem of MJSs was studied in [21].…”

Non‐zero‐sum games of discrete‐time Markov jump systems with unknown dynamics: An off‐policy reinforcement learning method

“…Then, this approach has been extended to various additional fields, for instance singularly perturbed systems, 37 nonlinear systems, 38 and MJSs. [39][40][41] Among them, the relevant research on MJSs attracts our attention. In Reference 39, a RL algorithm was given to solve the tracking control problem by augmenting the system and setting the appropriate cost function.…”

“…In Reference 36, a reinforcement learning (RL) algorithm was proposed to get rid of dependence on the system dynamics information by using the input and output data of the system. Then, this approach has been extended to various additional fields, for instance singularly perturbed systems, 37 nonlinear systems, 38 and MJSs 39‐41 . Among them, the relevant research on MJSs attracts our attention.…”

“…In Reference 39, a RL algorithm was given to solve the tracking control problem by augmenting the system and setting the appropriate cost function. The authors in Reference 40 decoupled the continuous‐time MJSs, then the corresponding online RL algorithm was designed to solve AREs by using the policy iteration method 42,43 . The above articles are mainly aimed at systems with a single controller, but these results cannot be directly applied to some systems that require multiple controllers to work together in real life, such as microgrid systems and thus on.…”