Model checking Markov Chains: Techniques and Tools

Abstract

Probabilistic model checking has been a successful research field in the recent decades. This dissertation deals with four important aspects of model checking Markov chains: the development of efficient model-checking tools, the improvement of model-checking algorithms, the efficiency of the state-space reduction techniques, and the development of simulation-based model-checking procedures.We start by introducing MRMC, a model checker for discrete-time and continuoustime Markov reward models. It supports reward extensions of PCTL and CSL, and allows for the automated verification of properties concerning long-run and instantaneous rewards as well as cumulative rewards. In particular, it supports to check the reachability of a set of goal states (by only visiting legal states before) under a time and an accumulated reward constraint. Several numerical algorithms and extensions thereof are included in MRMC. We study the efficiency of the tool in comparison with several probabilistic model checkers by comparing verification times and peak memory usage for a set of standard case studies. The study considers the model checkers E ⊢MC 2 , PRISM (sparse and hybrid mode), Ymer and VESTA, and focuses on fully probabilistic systems. Several of our experiments show significantly different run times and memory consumptions between the tools -up to various orders of magnitudewithout, however, indicating a clearly dominating tool. For statistical model checking, Ymer prevails whereas for the numerical tools MRMC and PRISM (sparse) are rather close.Further, we consider the time-bounded reachability problem for continuous-time Markov chains (CTMCs), the efficient algorithms for which are at the heart of probabilistic model checkers such as PRISM and E ⊢MC 2 . For large time spans, on-the-fly steady-state detection is commonly applied. To obtain correct results (up to a given accuracy), it is essential to avoid detecting premature stationarity. We give a detailed account of criteria for steady-state detection in the setting of time-bounded reachability. This is done for forward-and backward-reachability algorithms. As a spin-off of this study, new results for on-the-fly steady-state detection during CTMC transient analysis are reported. Based on these results, a precise procedure for steady-state detection for time-bounded reachability is obtained. Experiments show the impact of these results in probabilistic model checking.After that we study the effect of bisimulation minimization in model checking of monolithic discrete-and continuous-time Markov chains as well as variants thereof with rewards. Our results show that -as for traditional model checking -enormous state space reductions (up to logarithmic savings) may be obtained. While in traditional model checking, bisimulation minimisation pays off only rarely (because it is rather, we find often enough that the verification time of the original Markov chain exceeds the minimisation time plus the verification time of the reduced chain. We consider probabilistic bisimula...

Statistical Verification of Probabilistic Properties with Unbounded Until

“…Sampling is said to be unbiased if the expectation of the sample distribution is the same as the expectation of the true distribution. The use of unbiased sampling distinguishes our methods from most recent efforts to devise sampling-based algorithms for time-unbounded properties, which are based on biased sampling [21,18,27,3] (see Sect. 4).…”

“…This approach has been used in the past for program analysis [20], and more recently for model checking [27] using biased sampling. The algorithm is ensured termination for any finite-state homogeneous discrete-time Markov chain, although it is potentially applicable for any model for which we can perform reachability.…”

INFAMY: An Infinite-State Markov Model Checker

“…For several very large finite models, INFAMY is competitive with PRISM, as evident from Section 3. Model checkers based on discrete-event simulation [10,11,12,9] Using truncation, we compute a finite submodel which is sufficient for checking the validity of a property given in time-dependent CSL. This is done by descending into the CSL formula and at the same time exploring the model subject to the different time bounds given in the subformulas as well as the rates occurring in the model (details can be found in [13]).…”

The Ins and Outs of the Probabilistic Model Checker MRMC

“…MRMC implements an efficient algorithm to compute maximal time-bounded reachability probabilities in uniform CTMDPs [4]. Finally, (3) MRMC includes a discrete-event simulation engine for simulative CSL model checking [55]. MRMC is thus the first tool with simulation-based model checking of full CSL.…”