Computing Conditional Probabilities in Markovian Models Efficiently

Abstract

The fundamentals of probabilistic model checking for Markovian models and temporal properties have been studied extensively in the past 20 years. Research on methods for computing conditional probabilities for temporal properties under temporal conditions is, however, comparably rare. For computing conditional probabilities or expected values under ω-regular conditions in Markov chains, we introduce a new transformation of Markov chains that incorporates the effect of the condition into the model. For Markov decision processes, we show that the task to compute maximal reachability probabilities under reachability conditions is solvable in polynomial time, while it was conjectured to be computationally hard. Using adaptions of known automata-based methods, our algorithm can be generalized for computing the maximal conditional probabilities for ω-regular events under ω-regular conditions. The feasibility of our algorithms is studied in two benchmark examples.

Trade-off analysis meets probabilistic model checking

“…We addressed quantiles, conditional probabilities and ratio constraints for accumulated cost or reward functions for analyzing the interplay between multiple objectives. For quantiles and conditional probabilities, we already carried out prototype implementations [7,10] based on PRISM, which we could use in case studies from different domains (e.g., [9,25]). We are currently working on a prototype implementation in the context of ratios, involving several heuristics.…”

“…We summarize our recent results presented in [7,40] and briefly discuss the extension of quantiles towards conjunctive objectives. The transformation-based approach of [10] for the computation of conditional probabilities in Markovian models and the model-checking problem for conditional PCTL is described in Section 4. Reasoning about constraints on the ratio of the accumulated values of two reward functions and its relation to algorithmic problems for structures with an integervalued weight function is discussed in Section 5.…”

“…This is extremely helpful when analyzing exceptional system behavior of systems assuming that some very rare failure event occurred. Recently, it turned out that the computation of conditional probability queries can be done efficiently [10]. Quantiles and conditional probabilities can be combined very naturally.…”

A Storm is Coming: A Modern Probabilistic Model Checker

“…it provides a Python API facilitating easy and rapid prototyping of other tools using the engines and algorithms in Storm; -it provides the following functionalities under one roof: the synthesis of counterexamples and permissive schedulers (both MILP-and SMT-based), gamebased abstraction of infinite-state MDPs, efficient algorithms for conditional probabilities and rewards [9], and long-run averages on MDPs [10]; -its performance in terms of verification speed and memory footprint on the Prism benchmark suite is mostly better compared to Prism. [12].…”

“…To enable the treatment of reward objectives such as expected and long-run rewards, Storm supports reward extensions of these logics in a similar way as Prism. In addition, Storm supports conditional probabilities and conditional rewards [9]; these are, e.g., important for the analysis of cpGCL programs.…”