Hypergraph-based parallel computation of passage time densities in large semi-Markov models

Abstract

Passage time densities and quantiles are important performance and quality of service metrics, but their numerical derivation is, in general, computationally expensive. We present an iterative algorithm for the calculation of passage time densities in semi-Markov models, along with a theoretical analysis and empirical measurement of its convergence behaviour. In order to implement the algorithm efficiently in parallel, we use hypergraph partitioning to minimise communication between processors and to balance workloads. This enables the analysis of models with very large state spaces which could not be held within the memory of a single machine. We produce passage time densities and quantiles for very large semi-Markov models with over 15 million states and validate the results against simulation.

Uniformization and hypergraph partitioning for the distributed computation of response time densities in very large Markov models

“…Finally, in the example presented here several hundred successive matrix-vector multiplications are performed, but in other techniques for response time density calculation (e.g. [7]) many millions of such operations are performed, thus reducing the relative overhead of the hypergraph partitioning step.…”

Iterative convergence of passage-time densities in semi-Markov performance models

“…Having tested the iterative passage-time analysis technique on very small models, we demonstrate that it is scalable to large models by summarising analysis of a 15 million state SMP model of a distributed web server [5], shown in Fig. 5.…”

“…7 shows the density of the time taken to perform 100 reads and 50 page updates in the web server model. Calculation of the 35 t-points plotted required 2 days, 17 hours and 30 minutes using 64 slave processors [5]. Our algorithm evaluated L i j (s) at 1155 s-points, each of which involved manipulating sparse matrices of rank 15,445,919.…”

“…Techniques for evaluating passage-times in Markovian systems have existed for some time [2][3][4], however the practical calculation of passage-times in a semi-Markov environment, where arbitrary distributions of atomic events are permitted, is a recent development. In previous work [1,5] we described the iterative algorithm presented here, but had no direct proof that the iterative scheme converged to the correct passage-time. Here, we present a complete proof of convergence to the correct result.…”

Distributed computation of transient state distributions and passage time quantiles in large semi-Markov models

“…This builds upon our iterative technique for generating passage time densities and quantiles [1,2]. The algorithm is based on the calculation and subsequent numerical inversion of Laplace transforms.…”