Numerical Methods for Structured Markov Chains

Abstract

The book deals with the numerical solution of structured Markov chains which include M/G/1 and G/M/1-type Markov chains, QBD processes, non-skip-free queues, and tree-like stochastic processes and has a wide applicability in queueing theory and stochastic modeling. It presents in a unified language the most up to date algorithms, which are so far scattered in diverse papers, written with different languages and notation. It contains a thorough treatment of numerical algorithms to solve these problems, from the simplest to the most advanced and most efficient. Nonlinear matrix equations are at the heart of the analysis of structured Markov chains, they are analysed both from the theoretical, from the probabilistic, and from the computational point of view. The set of methods for solution contains functional iterations, doubling methods, logarithmic reduction, cyclic reduction, and subspace iteration, all are described and analysed in detail. They are also adapted to interesting specific queueing models coming from applications. The book also offers a comprehensive and self-contained treatment of the structured matrix tools which are at the basis of the fastest algorithmic techniques for structured Markov chains. Results about Toeplitz matrices, displacement operators, and Wiener-Hopf factorizations are reported to the extent that they are useful for the numerical treatment of Markov chains. Every and all solution methods are reported in detailed algorithmic form so that they can be coded in a high-level language with minimum effort.

Matrix structures and applications

“…If the zeros of a(x) and of b(x) lie in the unit disk this factorization is called Wiener-Hopf factorization. Wiener-Hopf factorizations are encountered in many applications, see for instance [52], [17].…”

“…Computing canonical Wiener-Hopf factorizations is fundamental in the solution of many queuing models [17]. An example of application to queueing model is given by the shortest queue problem.…”

“…., of length m. In fact, assuming that π (0) is known, then the computation of π (i) for i > 0 is reduced to solving two block triangular Toeplitz systems. The computation of π (0) can be performed by using suitable formulas [17].…”

A fast algorithm of two-level banded Toeplitz systems of linear equations with application to image restoration

“…In fact, in the Step 2 which appears in both the block Toeplitz matrix-vector products can be computed with the block-DFT (Block Discrete Fourier Transform) in O(m 2 n log n + m 3 n) (For more details, see [5]). …”

The mean queue content of discrete-time queues with zero-regenerative arrivals

“…Prime examples of the latter class include M/G/∞-input -often referred to as train arrival or session models, see [3,5,13] -as well as autoregressive arrival models [7,8,10]. Most often, performance analysis of queues with finite-state-space Markovian arrivals relies on a computational approach: efficient algorithms are devised to calculate the performance measures of interest [4]. In contrast, for particular types of autoregressive arrival models and train arrival models, closed-form expressions for the first moment or the first few moments of the queue content and delay are available.…”