Generalized semi-Markov decision processes

Doshi, Bharat T.

doi:10.2307/3213089

Cited by 10 publications

(3 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A policy (as in Markov Decision Processes [16]) picks an event in the set of enabled controllable events of a state to generate a partial execution path. Thereafter, the optimal strategy over different policy executions over a GSMDP can be obtained using the techniques of [15,17].…”

Section: Definition 1 a Generalised Semi-markov Decisionmentioning

confidence: 99%

“…In Section 2 we introduce the syntax of the Stochastic Quality Calculus. The operational semantics of processes is presented in Section 3; it makes use of general distributions and in Section 4 we show that it amounts to Generalised Semi-Markov Decision Processes (GSMDPs) [15] by modelling truly concurrent broadcast communication as discrete events. Some of the discrete events in GSMDPs are controllable, which introduces a decision dimension to execute controllable events nondeterministically; a policy (as in Markov decision processes [16]) is introduced to deal with the nondeterminism.…”

Section: Introductionmentioning

confidence: 99%

“…Some of the discrete events in GSMDPs are controllable, which introduces a decision dimension to execute controllable events nondeterministically; a policy (as in Markov decision processes [16]) is introduced to deal with the nondeterminism. The classical problem is then to find an optimal strategy to maximise some reward function on GSMDPs and we refer to [15,17] for the analysis. In order to avoid the decision dimension and perform purely stochastic reasoning on the systems, in Section 5 we introduce two analysable fragments of SQC that both admit truly concurrent behaviour.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Stochastic Quality Calculus

Zeng¹,

Nielson²,

Nielson³

2014

Lecture Notes in Computer Science

View full text Add to dashboard Cite

International audienceWe introduce the Stochastic Quality Calculus in order to model and reason about distributed processes that rely on each other in order to achieve their overall behaviour. The calculus supports broadcast communication in a truly concurrent setting. Generally distributed delays are associated with the outputs and at the same time the inputs impose constraints on the waiting times. Consequently, the expected inputs may not be available when needed and therefore the calculus allows to express the absence of data.The communication delays are expressed by general distributions and the resulting semantics is given in terms of Generalised Semi-Markov Decision Processes. By restricting the distributions to be continuous and by allowing truly concurrent communication we eliminate the non-determinism and arrive at Generalised Semi-Markov Processes (GSMPs); further restriction to exponential distributions gives rise to numerically analysable GSMPs, in particular using techniques from stochastic model checking

show abstract

Section: Definition 1 a Generalised Semi-markov Decisionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation