Methods to design optimal control of Markov process with finite state set in the presence of constraints

Miller, Bruce W.; Miller, Gregory; Semenikhin, K. V.

doi:10.1134/s000511791102010x

Cited by 22 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Proof: For a detailed discussion see [20] and [21]. Note that this includes the definition of consistency given in [19], that is, that there exists some feasible (C(t, p(t)), ν(t)) such that J 1 − α 2 ≤ 0 and J 2 − β 2 ≤ 0, but is stronger in the sense that this holds for any convex combination of the criteria.…”

Section: A Consistency Of Constraintsmentioning

confidence: 97%

Optimal control of time-inhomogeneous Markov chains with application to dam management

McInnes

Miller

2013

2013 Australian Control Conference

View full text Add to dashboard Cite

We consider a time-inhomogeneous Markov chain with a compound Poisson process as an input as an important approximation to get the solution of real problems. Outflows are comprised of both controlled and uncontrolled counting processes. We demonstrate the utility of this model in the specific problem of demand control and flood prevention in a nearly full dam, where the dam is modeled as a continuous-time controllable Markov chain under control resource constraints. This work significantly extends previous results because the inflow of rain is potentially sudden and of large volume, so a compound Poisson process is preferred to simple counting processes. On the other hand the safe release of water is constrained and so is modelled as a simple counting process. A proof of the infinitesimal generator of the Markov chain with these characteristics is given and a numerical example demonstrates the effectiveness of the controls.

show abstract

Section: A Consistency Of Constraintsmentioning

confidence: 97%

Optimal control of time-inhomogeneous Markov chains with application to dam management

McInnes

Miller

2013

2013 Australian Control Conference

View full text Add to dashboard Cite

show abstract

“…resulting the row of equal stationary state probabilities (1, 1, 1, 1, 1, 1, 1)/7 ; therefore, parameters β i coincide due to both (25) and (26). The mean loss matrix is as follows having the minimum mean losses A min = 0.1482 ; therefore, parameter σ = 1, and optimal pure strategy is represented by vector of integers * = (1, 2, 3, 4, 1, 1, 1) T .…”

Section: Numerical Examplementioning

confidence: 97%

“…During recent years Controlled Markov Chains (CMC) become the useful mathematical tool applicable to the solution of stochastic optimal control problems with constraints [18], [24], [25]. Models of CMC come from stock and production management [6], [12], internet congestion control [17], large dams management [22], [23] and many others.…”

Section: Introductionmentioning

confidence: 99%

On effectiveness of the Mirror Decent Algorithm for a stochastic multi-armed bandit governed by a stationary finite Markov chain

Nazin

Miller

2013

2013 Australian Control Conference

View full text Add to dashboard Cite

In this article, we study the effectiveness of the Mirror Descent Randomized Control Algorithm recently developed to a class of homogeneous finite Markov chains governed by the stochastic multi-armed bandit with unknown mean losses. We prove the explicit, non-asymptotic both upper and lower bounds for the mean losses at a given (finite) time horizon. These bounds are very similar as functions of problem parameters and time horizon, but with different logarithmic term and absolute constant. Numerical example illustrates theoretical results.

show abstract

Two-phase queueing system optimization in applications to data transmission control

Kuznetsov

Myasnikov

Semenikhin

2017

Procedia Engineering

View full text Add to dashboard Cite

Methods to design optimal control of Markov process with finite state set in the presence of constraints

Cited by 22 publications

References 21 publications

Optimal control of time-inhomogeneous Markov chains with application to dam management

Optimal control of time-inhomogeneous Markov chains with application to dam management

On effectiveness of the Mirror Decent Algorithm for a stochastic multi-armed bandit governed by a stationary finite Markov chain

Two-phase queueing system optimization in applications to data transmission control

Contact Info

Product

Resources

About