On Stability and LQ Control of MJLS With a Markov Chain With General State Space

Abstract

We study the mean square stability and the LQ control of discrete time Markov Jump Linear Systems where the Markov chain has a general state space. The mean square stability is characterized by the spectral radius of an operator describing the evolution of the second moment of the state vector. Two equivalent tests for the mean square stability are obtained based on the existence of a positive definite solution to a Lyapunov equation and a uniformity result respectively. An algorithm for testing the mean square stability is also developed based on the uniformity result. The finite and infinite horizon LQ problems are considered and their solutions are characterized by appropriate Riccati integral equations. An application to Networked Control Systems (NCS) is finally presented and a simple example is studied via simulation.Index Terms-Markov jump linear systems, stochastic optimal control, stochastic stability.

Discrete-time jump linear systems with Markov chain in a general state space.

“…The main result presented in Appendix A.1 states that, under some conditions based on the concepts of (stochastic) stabilizability and detectability, there exists a unique (in the µ-almost everywhere sense) stabilizing positive-semidefinite solution for the control S-coupled algebraic Riccati equations and, as shown in Section 5.2, it provides a solution to the infinite-horizon LQ optimal control problem. Notice that these results, based on the concepts of (stochastic) stabilizability and detectability, are substantially different from the results obtained by Kordonis and Papavassilopoulos (2014).…”

“…Only more recently the general state space case, in which the Markov chain takes values in a general state space, started to be analyzed. Regarding some stability results it can be mentioned the papers by Li et al (2012) and Kordonis and Papavassilopoulos (2014), in which the exponential almost sure stability and mean square stability were considered. Assuming that the Markov chain is a positive Harris chain, Li et al (2012) showed in Theorem 4.3 that the uniform exponentially almost sure stability is equivalent to a contractivity condition being satisfied.…”

“…Ungureanu et al (2013), Ungureanu (2009) and Costa et al (2005b) have also dealt with the LQ optimal control of discrete-time MJLS with Markov chain in an infinite countable state space. Kordonis and Papavassilopoulos (2014) have characterized the finite and infinitehorizon LQ optimal control problems for MJLS with Markov chain in a general state space in terms of appropriate Riccati integral equations. Kordonis and Papavassilopoulos (2014) obtained that if there exists a police that yields a finite cost for the infinite-horizon LQ optimal control problem then the optimal cost has a quadratic form, written in terms of a solution to a Riccati integral equation, and the optimal control law is in a state feedback form, and, conversely, if there exists a solution to the Riccati integral equation and the feedback control law derived from this solution yields a mean square exponentially stable closed loop system, then this solution is optimal for the infinite-horizon LQ optimal control problem.…”

LQ-Optimal Sampled-Data Control under Stochastic Delays: Gridding Approach for Stabilizability and Detectability

“…In practice, communication delays take a continuum of values and need to be modeled by more general stochastic processes such as Markov processes in [3,15,16]. A notable exception can be found in [23], in which the authors have presented an offline computation method of delay-dependent LQ controllers for systems with continuous-valued Markovian delays. The formulation in [23] requires a solution of a nonlinear vector integral equation called the Riccati integral equation and ignores the intersample behavior of the closed-loop system.…”

“…By definition, the matrices A and B satisfy A ∈ H n sup and B ∈ H n×m sup . This delaydependent discrete-time system is widely used for the analysis of time-delay systems, e.g., in [9,10,18,23,26,27,34].…”