Markov Models and Optimization

Davis, Mark H. A.

doi:10.1007/978-1-4899-4483-2

Cited by 983 publications

(1,119 citation statements)

References 0 publications

Supporting

Mentioning

1,107

Contrasting

Unclassified

Order By: Relevance

“…Consequently, the natural class of limits will by hybrid or piecewise deterministic (in the sense of Davis [11]) models in which some components are discrete and some are absolutely continuous. See Section 3 of [4] and Section 6.3 of [26] for examples.…”

Section: Hybrid Limitsmentioning

confidence: 99%

Continuous Time Markov Chain Models for Chemical Reaction Networks

Anderson¹,

Kurtz

2011

Design and Analysis of Biomolecular Circuits

255

306

View full text Add to dashboard Cite

A reaction network is a chemical system involving multiple reactions and chemical species. The simplest stochastic models of such networks treat the system as a continuous time Markov chain with the state being the number of molecules of each species and with reactions modeled as possible transitions of the chain. This chapter is devoted to the mathematical study of such stochastic models. We begin by developing much of the mathematical machinery we need to describe the stochastic models we are most interested in. We show how one can represent counting processes of the type we need in terms of Poisson processes. This random time-change representation gives a stochastic equation for continuoustime Markov chain models. We include a discussion on the relationship between this stochastic equation and the corresponding martingale problem and Kolmogorov forward (master) equation. Next, we exploit *

show abstract

Section: Hybrid Limitsmentioning

confidence: 99%

Continuous Time Markov Chain Models for Chemical Reaction Networks

Anderson¹,

Kurtz

2011

Design and Analysis of Biomolecular Circuits

255

306

View full text Add to dashboard Cite

show abstract

“…A suitable mathematical framework for defining our model could be that of Piece-wise Deterministic Markov Processes (PDMPs) [15], where continuous flows defined as systems of ODEs are reset by stochastic jumps distributed according to exponential distributions. Therefore, one would naturally associate changes in the random environment with such jumps, and the queueing network dynamics within one stage as a continuous flow by the fluid model.…”

Section: Related Workmentioning

confidence: 99%

Blending randomness in closed queueing network models

Casale

Tribastone

Harrison

2014

Performance Evaluation

View full text Add to dashboard Cite

Random environments are stochastic models used to describe events occurring in the environment a system operates in. The goal is to describe events that affect performance and reliability such as breakdowns, repairs, or temporary degradations of resource capacities due to exogenous factors. Despite having been studied for decades, models that include both random environments and queueing networks remain difficult to analyse. To cope with this problem, we introduce the blending algorithm, a novel approximation for closed queueing network models in random environments. The algorithm seeks to obtain the stationary solution of the model by iteratively evaluating the dynamics of the system in between state changes of the environment. To make the approach scalable, the computation relies on a fluid approximation of the queueing network model. A validation study on 1,800 models shows that blending can save a significant amount of time compared to simulation, with an average accuracy that grows with the number of servers in each station. We also give an interpretation of this technique in terms of Laplace transforms and use this approach to determine convergence properties.

show abstract

“…It is known that X can be endowed with a metric ρ whose restriction to any component X i is equivalent to the natural Euclidean metric of this component Davis [1993]. Then (X, B(X)) is a Borel space (homeomorphic to a Borel subset of a complete separable metric space), where B(X) is the Borel σ-algebra of X.…”

Section: The Computational Modelmentioning

confidence: 99%

“…The meaning of the elements of M can be found in any source treating continuous-parameter Markov processes (for e.g. Blumenthal et al [1968] or Ethier et al [1986] or Davis [1993]). We adjoin an extra point ∆ (the cemetery) to X as an isolated point,…”

Section: The Computational Modelmentioning

confidence: 99%

State constrained reachability for stochastic hybrid systems

Bujorianu

2011

Nonlinear Analysis: Hybrid Systems

View full text Add to dashboard Cite

The stochastic hybrid systems constitute well established classes of realistic models of hybrid discrete/continuous dynamics subject to random perturbations, autonomous uncontrollable transitions, nondeterminism or uncertainty. Stochastic reachability analysis is a key factor in the verification and deployment of stochastic hybrid systems. The encouraging recent progress prompts us to refine the problem to cover more realistic situations. We extend the so called constrained reachability problem from the probabilistic discrete case to stochastic hybrid systems. Then we define mathematically this problem, and we obtain the reach probabilities as solutions of a boundary value problem. The last problem is well studied and numerical, even symbolic solutions exist. This characterization is useful in stochastic control, in probabilistic path planning and for nano-systems.

show abstract

Markov Models and Optimization

Cited by 983 publications

References 0 publications

Continuous Time Markov Chain Models for Chemical Reaction Networks

Continuous Time Markov Chain Models for Chemical Reaction Networks

Blending randomness in closed queueing network models

State constrained reachability for stochastic hybrid systems

Contact Info

Product

Resources

About