Computing Stationary Expectations in Level-Dependent QBD Processes

Abstract

Stationary expectations corresponding to long-run averages of additive functionals on level-dependent quasi-birth-and-death processes are considered. Special cases include long-run average costs or rewards, moments and cumulants of steady-state queueing network performance measures, and many others. We provide a matrix-analytic scheme for numerically computing such stationary expectations without explicitly computing the stationary distribution of the process, which yields an algorithm that is as quick as its counterparts for stationary distributions but requires far less computer storage. Specific problems arising in the case of infinite state spaces are discussed and the application of the algorithm is demonstrated by a queueing network example.

Bounded truncation error for long-run averages in infinite Markov chains

“…Error bounds for long-run averages rather than for probabilities are particularly valuable when we have a method available that computes long-run averages without explicitly relying on the stationary distribution (if it exists) or an invariant measure. In particular, the state space truncation method solves the open issue that the memory-efficient matrixanalytic method presented in [5] for computing stationary expectations in LDQBD processes without at first explicitly computing the stationary distribution was lacking an accuracy measure. Now, in conjunction with the state space truncation method of this paper, [5] constitutes a powerful matrix-analytic method for numerically approximating long-run averages of additive functionals in infinite recurrent LDQBD processes, where an approximation error bound can be specified a priori.…”

“…In particular, the state space truncation method solves the open issue that the memory-efficient matrixanalytic method presented in [5] for computing stationary expectations in LDQBD processes without at first explicitly computing the stationary distribution was lacking an accuracy measure. Now, in conjunction with the state space truncation method of this paper, [5] constitutes a powerful matrix-analytic method for numerically approximating long-run averages of additive functionals in infinite recurrent LDQBD processes, where an approximation error bound can be specified a priori. This enormously advances the state of the art in matrix-analytic computations and their applicability to, e.g.…”

“…Think, for example, of infinite LDQBD processes, where the states are ordered according to the chosen level definition and the block structured generator matrix is truncated at certain blocks corresponding to high (or low) level numbers such as, e.g. in [3], [4], [5], [7], [9], and [15]. Then it is often more convenient to consider an appropriate finite superset of C rather than to work directly with C. For instance, simple algebra yields…”

Inherent Numerical Instability in Computing Invariant Measures of Markov Chains

“…For practical issues, it would be desirable to find a memory-efficient algorithm for computing stationary expectations. For quasi-birth-death processes (that is, block-tridiagonal transition matrices), such a method has been suggested in [34].…”