Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes

Abstract

Abstract. This paper introduces strong bisimulation for continuous-time Markov decision processes (CTMDPs), a stochastic model which allows for a nondeterministic choice between exponential distributions, and shows that bisimulation preserves the validity of CSL. To that end, we interpret the semantics of CSL-a stochastic variant of CTL for continuous-time Markov chains-on CTMDPs and show its measure-theoretic soundness. The main challenge faced in this paper is the proof of logical preservation that is substantially based on measure theory.

Time-Bounded Reachability Probabilities in Continuous-Time Markov Decision Processes

“…The reachability analysis is the key ingredient to enable approximate model checking of CTMDPs with respect to logics like CSL [10]. Note however, that for CSL, it is necessary to consider time-interval (instead of timebounded) reachability.…”

Bisimulations and Logical Characterizations on Continuous-Time Markov Decision Processes

“…Thus, as for other stochastic models, bisimulation relations have been proposed for CTMDPs. In [27], strong bisimulation was shown to be sound with respect to the continuous-time stochastic logic [2] (CSL). This result guarantees that one can first reduce a CTMDP up to bisimulation equivalence before analysing it.…”

“…This result guarantees that one can first reduce a CTMDP up to bisimulation equivalence before analysing it. On the other hand, as indicated in [27], strong bisimulation is not complete with respect to CSL, i.e., logically equivalent states might be not bisimilar.…”

“…We start with a slightly coarser notion of strong bisimulation than the one in [27], and then propose weak bisimulation for CTMDPs. We study the relationship between strong and weak bisimulations and the logical equivalences induced by CSL and CSL \X -the sub-logic of CSL without next operators.…”

“…In [18], the results are extended to CTMCs with continuous state spaces. For CTMDPs, the first logical characterization result is presented in [27]. It is shown that strong bisimulation is sound, but not complete with respect to CSL equivalence.…”