Model-free adaptive optimal control policy for Markov jump systems: A value iterations algorithm

Abstract

This article develops a model-free adaptive optimal control policy for discrete-time Markov jump systems. First, a two-player zero-sum game is formulated to obtain an optimal control policy that minimizes a cost function against the worst-case disturbance. Second, an action and mode-dependent value function is set up for zero-sum game to search such a policy with convergence guarantee rather than solving an optimization problem satisfying coupled algebraic Riccati equations. To be specific, motivated by the Bellman optimal principle, we develop an online value iterations algorithm to solve the zero-sum game, which is learning while controlling without any initial stabilizing policy. By this algorithm, we can achieve disturbance attenuation for Markov jump systems without knowledge of the system matrices. The adaptivity to slowly changing uncertainties can also be achieved due to the model-free feature and policy convergence. Finally, the effectiveness and practical potential of the algorithm are demonstrated by considering two numerical examples and a solar boiler system.

Asynchronous control for Markov jump systems subject to actuator saturation and partial mode information

“…4 The synchronous neural network-based adaptive fault-tolerant control was considered for MJSs with nonlinearity and actuator faults. 5 A model-free adaptive optimal control policy for discrete-time MJSs was developed in the works of Zhou et al 6 The H ' control problem of a class of singular MJSs with timevarying inputs and state delays was investigated. 7 Furthermore, in the control field, actuator saturation is an undesired phenomenon, which results from physical limitations and may degrade the performance of systems, even cause systems to become unstable.…”

“…4 The synchronous neural network-based adaptive fault-tolerant control was considered for MJSs with nonlinearity and actuator faults. 5 A model-free adaptive optimal control policy for discrete-time MJSs was developed in the works of Zhou et al 6 The H ∞ control problem of a class of singular MJSs with time-varying inputs and state delays was investigated. 7…”