Uniformity by Construction in the Analysis of Nondeterministic Stochastic Systems

Abstract

Continuous-time Markov decision processes (CTMDPs) are behavioral models with continuous-time, nondeterminism and memoryless stochastics. Recently, an efficient timed reachability algorithm for CTMDPs has been presented [2], allowing one to quantify, e. g., the worst-case probability to hit an unsafe system state within a safety critical mission time. This algorithm works only for uniform CTMDPs -CTMDPs in which the sojourn time distribution is unique across all states. In this paper we develop a compositional theory for generating CTMDPs which are uniform by construction. To analyze the scalability of the method, this theory is applied to the construction of a fault-tolerant workstation cluster example, and experimentally evaluated using an innovative implementation of the timed reachability algorithm. All previous attempts to modelcheck this seemingly well-studied example needed to ignore the presence of nondeterminism, because of lacking support for modelling and analysis. MotivationNondeterministic stochastic systems combine the behavior of classical labeled transition systems (LTSs), with classical stochastic processes, Markov chains in particular. The most widely known model is that of Markov decision processes in discrete time (DTMDPs) which combines LTS and discretetime Markov chains (DTMCs). This model originates from the operations research (OR) and planning context. In the setting of concurrency theory, that model is often called probabilistic automata [27,28], and has been the subject of many studies [24,22]. Model checking algorithms for DTMDPs are available [3,7], have been implemented and applied [18].Continuous-time Markov chains (CTMCs) extend DTMCs with memoryless continuous time. These models have a very rich spectrum of applications, ranging from disk storage dimensioning [26] to signalling transduction networks [6]. In the context of concurrency theory, the extension of CTMCs with nondeterminism has lead to different models, including stochastic transition systems [8] and interactive Markov chains (IMC) [14]. The latter is an orthogonal superposition of LTSs and CTMCs with convenient compositional properties.On the other hand, continuous-time Markov decision processes (CTMDPs) have been proposed in OR [25], but not many results are available, and model checking of CTMDP has not been addressed successfully thus far. A core ingredient to model check timed safety and liveness properties has been developed in [2] by an algorithm addressing timed-bounded reachability probabilities in a CTMDP. The algorithm computes the maximal probability to reach a set of goal states within a given time bound, among all CTMCs induced by timeabstract schedulers. Unfortunately, the algorithm is of limited generality, since it is restricted to so-called uniform CTMDPs (uCTMDPs), CTMDPs in which the sojourn time distribution is unique across all states.Nevertheless we have recently made use of this algorithm in the verification of large STATEMATE [4] models of train control systems. The algorithm allowed us to verify...

A Compositional Semantics for Dynamic Fault Trees in Terms of Interactive Markov Chains

“…In the absence of simultaneous failures [4] in the DFT model, the algorithm results in an aggregated CTMC. However, in cases with simultaneous failures the result can be a continuous-time Markov decision process, which can be analyzed by computing bounds on the performance measure of interest [2,15].…”

A Tutorial on Interactive Markov Chains

“…Note that a CTMDP combines the two transition relations of an IMC in one transition rate matrix. We recapitulate a transformation from an IMC to a CT-MDP [40,38,37] which preserves important properties of the original IMC, and thus can be used to apply CTMDP analysis techniques [16,6] on the transformed model.…”

A uniform definition of stochastic process calculi

“…The structure of Continuous Time Markov Decision Processes (CTMDPs), as defined by Hermanns and Johr [Hermanns and Johr 2007], is similar to the structure of finite support FuTSs with action-indexed random delays (∆ A -labeled FsFuTS over R ≥0 ). Indeed, a CTMDP is a tuple M = (S , A, T, s 0 ) with S a (finite) set of states, A a (finite) set of action labels, s 0 ∈ S , and T ⊆ S × A × FTF(S , R ≥0 ) the transition relation.…”