Random-Time, State-Dependent Stochastic Drift for Markov Chains and Application to Stochastic Stabilization Over Erasure Channels

Abstract

It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of stochastic networks. In this paper we extend the general theory to randomized multi-step Lyapunov theory to obtain criteria for stability and steady-state performance bounds, such as finite moments.These results are applied to a remote stabilization problem, in which a controller receives measurements from an erasure channel with limited capacity. Based on the general results in the paper it is shown that stability of the closed loop system is assured provided that the channel capacity is greater than the logarithm of the unstable eigenvalue, plus an additional correction term. The existence of a finite second moment in steady-state is established under additional conditions.

Design of Information Channels for Optimization and Stabilization in Networked Control

“…We note that for erasure channels and noiseless channels, one can obtain tight converse theorems using Theorem 6.18 (see [36] and [64]). For general DMCs, however, a tight converse result on quadratic stabilizability is not yet available.…”

“…We will see that under a more generalized interpretation of stationarity, this result applies to a large class of memoryless channels and a class of channels with memory as to be seen later in this chapter (see Theorem 6.27): There is a direct relationship between the existence of a stationary measure and the Shannon capacity of the channel used in the system. Under slightly stronger conditions we obtain a finite second moment: Theorem 6.23 [64] Suppose that the assumptions of Theorem 6.21 hold, and in addition the following bound holds:…”

“…Particularly related references include [11, 28-30, 32, 37, 42, 43, 45, 46, 54, 55, 59]. In the context of discrete channels, many of these papers considered a bounded noise assumption, except notably [30,36,37,55,64], and [56]. We refer the reader to [32] and [60] for a detailed literature review.…”

“…In this section, we will present a proof program developed in [55] and [64] for stochastic stabilization of Markov chains with event-driven samplings applied to networked control. Toward this end, we will first review few results from the theory of Markov chains.…”

“…In many practical settings, compact sets are small; sufficient conditions on when a compact set is small has been presented in [60] and [35]. Theorem 6.20 [64] Suppose that X is a ϕ-irreducible and aperiodic Markov chain. Suppose moreover that there are functions V :…”