On Probabilistic Parallel Programs with Process Creation and Synchronisation

Kiefer, Stefan; Wojtczak, Dominik

doi:10.1007/978-3-642-19835-9_28

Cited by 7 publications

(7 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The probability of acceptance of a run of a BMC by a deterministic parity tree automaton was studied in [4] and shown to be computable in PSPACE and in polynomial time for probabilities 0 or 1. In [16] a generalisation of the BMCs was considered that allowed for limited synchronisation of different tasks.…”

Section: Related Workmentioning

confidence: 99%

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Hahn

Perez

Schewe

et al. 2021

Computer Aided Verification

Self Cite

View full text Add to dashboard Cite

We study reinforcement learning for the optimal control of Branching Markov Decision Processes (BMDPs), a natural extension of (multitype) Branching Markov Chains (BMCs). The state of a (discrete-time) BMCs is a collection of entities of various types that, while spawning other entities, generate a payoff. In comparison with BMCs, where the evolution of a each entity of the same type follows the same probabilistic pattern, BMDPs allow an external controller to pick from a range of options. This permits us to study the best/worst behaviour of the system. We generalise model-free reinforcement learning techniques to compute an optimal control strategy of an unknown BMDP in the limit. We present results of an implementation that demonstrate the practicality of the approach.

show abstract

Section: Related Workmentioning

confidence: 99%

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Hahn

Perez

Schewe

et al. 2021

Computer Aided Verification

Self Cite

View full text Add to dashboard Cite

show abstract

“…[12,2]. In [13] an extension of branching processes was introduced to model parallel programs with stochastic process creation. The broad applicability of branching processes arises from their simplicity: each type models a class of threads (or tasks, animals, infections, molecules, grammatical structures) with the same probabilistic behaviour.…”

Section: − − → (B B)mentioning

confidence: 99%

“…[12,2] and the references therein) does not consider model-checking problems. Probabilistic split-join systems [13] are branching processes with additional features for process synchronisation and communication. The paper [13] focuses on performance measures (such as runtime, space and work), and does not provide a functional analysis.…”

Section: − − → (B B)mentioning

confidence: 99%

“…Probabilistic split-join systems [12] are branching processes with additional features for process synchronisation and communication. The paper [12] focuses on performance measures (such as runtime, space and work), and does not provide a functional analysis. The models of pPDAs and RMCs also feature dynamic task creation by means of procedure calls, however, as discussed above, the existing modelchecking algorithms [7,9,10] do not work for branching processes.…”

Section: Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Model Checking Stochastic Branching Processes

Chen

Dräger

Kiefer

2012

Mathematical Foundations of Computer Science 2012

Self Cite

View full text Add to dashboard Cite

Abstract. Stochastic branching processes are a classical model for describing random trees, which have applications in numerous fields including biology, physics, and natural language processing. In particular, they have recently been proposed to describe parallel programs with stochastic process creation. In this paper, we consider the problem of model checking stochastic branching process. Given a branching process and a deterministic parity tree automaton, we are interested in computing the probability that the generated random tree is accepted by the automaton. We show that this probability can be compared with any rational number in PSPACE, and with 0 and 1 in polynomial time. In a second part, we suggest a tree extension of the logic PCTL, and develop a PSPACE algorithm for model checking a branching process against a formula of this logic. We also show that the qualitative fragment of this logic can be model checked in polynomial time.

show abstract

“…While these models allow for a probabilistic splitting of tasks by pushing new procedures on a stack, the produced tasks are processed in a strictly sequential manner, whereas the queues in a queueing network process jobs in parallel and in continuous time. Recently, probabilistic split-join systems were introduced [17], which allow for branching but not for external arrivals, and assume unlimited parallelism. In [18, chapter 8] a queueing model with multiple classes of tasks and "feedback" is discussed, which is similar to our branching except that there is only one server, hence there is no parallelism.…”

Section: Introductionmentioning

confidence: 99%

Stabilization of Branching Queueing Networks

Brázdil,

Kiefer

2011

Preprint

Self Cite

View full text Add to dashboard Cite

Queueing networks are gaining attraction for the performance analysis of parallel computer systems. A Jackson network is a set of interconnected servers, where the completion of a job at server i may result in the creation of a new job for server j. We propose to extend Jackson networks by "branching" and by "control" features. Both extensions are new and substantially expand the modelling power of Jackson networks. On the other hand, the extensions raise computational questions, particularly concerning the stability of the networks, i.e, the ergodicity of the underlying Markov chain. We show for our extended model that it is decidable in polynomial time if there exists a controller that achieves stability. Moreover, if such a controller exists, one can efficiently compute a static randomized controller which stabilizes the network in a very strong sense; in particular, all moments of the queue sizes are finite.

show abstract

On Probabilistic Parallel Programs with Process Creation and Synchronisation

Cited by 7 publications

References 27 publications

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Model-Free Reinforcement Learning for Branching Markov Decision Processes

Model Checking Stochastic Branching Processes

Stabilization of Branching Queueing Networks

Contact Info

Product

Resources

About