A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes

Wolovick, Nicolás; Johr, Sven

doi:10.1007/11867340_25

Cited by 29 publications

(40 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We conjecture that our transformation is also measure-preserving for TTH and TH schedulers. Finally, late schedulers are shown to be able to improve upon generic schedulers [6].…”

Section: Resultsmentioning

confidence: 99%

“…This hierarchy refines the notion of generic measurable schedulers [6]. An important distinguishing criterion is the level of detail of timing information the schedulers may exploit, e.g., the delay in the last state, total time (TT), or all individual state residence times (T).…”

Section: Introductionmentioning

confidence: 87%

“…As proved in [6], the most general class is the set of all gm-schedulers: If paths π 1 , π 2 ∈ Paths ⋆ of a CTMDP end in state s, a gm-scheduler D :…”

Section: The Probability Spacementioning

confidence: 99%

See 2 more Smart Citations

Delayed Nondeterminism in Continuous-Time Markov Decision Processes

Neuhäußer

Stoelinga

Katoen

2009

Foundations of Software Science and Computational Structures

View full text Add to dashboard Cite

Abstract. Schedulers in randomly timed games can be classified as to whether they use timing information or not. We consider continuous-time Markov decision processes (CTMDPs) and define a hierarchy of positional (P) and historydependent (H) schedulers which induce strictly tighter bounds on quantitative properties on CTMDPs. This classification into time abstract (TA), total time (TT) and fully time-dependent (T) schedulers is mainly based on the kind of timing details that the schedulers may exploit. We investigate when the resolution of nondeterminism may be deferred. In particular, we show that TTP and TAP schedulers allow for delaying nondeterminism for all measures, whereas this does neither hold for TP nor for any TAH scheduler. The core of our study is a transformation on CTMDPs which unifies the speed of outgoing transitions per state.

show abstract

“…We conjecture that our transformation is also measure-preserving for TTH and TH schedulers. Finally, late schedulers are shown to be able to improve upon generic schedulers [6].…”

Section: Resultsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 87%

See 1 more Smart Citation

Delayed Nondeterminism in Continuous-Time Markov Decision Processes

Neuhäußer

Stoelinga

Katoen

2009

Foundations of Software Science and Computational Structures

View full text Add to dashboard Cite

show abstract

“…As pointed out in [17], not all schedulers are meaningful, even in the restricted case of continuous-time Markov decision processes (CTMDPs). In particular, under some schedulers, the set of runs reaching a given location can be non-measurable.…”

Section: Schedulers For Tamdpsmentioning

confidence: 99%

Playing Optimally on Timed Automata with Random Delays

Bertrand¹,

Schewe²

2012

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. We marry continuous time Markov decision processes (CTMDPs) with stochastic timed automata into a model with joint expressive power. This extension is very natural, as the two original models already share exponentially distributed sojourn times in locations. It enriches CTMDPs with timing constraints, or symmetrically, stochastic timed automata with one conscious player. Our model maintains the existence of optimal control known for CTMDPs. This also holds for a richer model with two players, which extends continuous time Markov games. But we have to sacrifice the existence of simple schedulers: polyhedral regions are insufficient to obtain optimal control even in the single-player case.

show abstract

“…Although this result is perhaps not surprising, its proof is non-trivial and strongly relies on measure-theoretic aspects. It shows that reasoning about CTMDPs, as witnessed also by [30,7,10] is not straightforward. As for MDPs, CSL equivalence does not coincide with bisimulation as only maximal and minimal probabilities can be logically expressed.…”

Section: Introductionmentioning

confidence: 98%

Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes

Neuhäußer

Katoen

CONCUR 2007 – Concurrency Theory

View full text Add to dashboard Cite

Abstract. This paper introduces strong bisimulation for continuous-time Markov decision processes (CTMDPs), a stochastic model which allows for a nondeterministic choice between exponential distributions, and shows that bisimulation preserves the validity of CSL. To that end, we interpret the semantics of CSL-a stochastic variant of CTL for continuous-time Markov chains-on CTMDPs and show its measure-theoretic soundness. The main challenge faced in this paper is the proof of logical preservation that is substantially based on measure theory.

show abstract

A Characterization of Meaningful Schedulers for Continuous-Time Markov Decision Processes

Cited by 29 publications

References 12 publications

Delayed Nondeterminism in Continuous-Time Markov Decision Processes

Delayed Nondeterminism in Continuous-Time Markov Decision Processes

Playing Optimally on Timed Automata with Random Delays

Bisimulation and Logical Preservation for Continuous-Time Markov Decision Processes

Contact Info

Product

Resources

About