First Passage Problems for Nonstationary Discrete-Time Stochastic Control Systems

Guo, Xianping; Hernández-del-Valle, Adrián; Hernández–Lerma, Onésimo

doi:10.3166/ejc.18.528-538

Cited by 15 publications

(16 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Among of the researches on the discounted DTMDPs, most literature such as [1,2,6,9,14,15,21,22] shared a common feature: the discount factor is a constant, which obviously cannot cover the important case of non-constant interest rates in a bank, and which motivates the studies on the general case of varying discount factors. For example, [8] studied the first passage problem for non-stationary nonlinear discrete-time stochastic control systems with the time-and state-dependent discount factors, and proved the existence of an optimal Markov policy under suitable conditions. Recently, [24] considered the first passage model of discounted DTMDPs in Borel state and action spaces with state-dependent discount factors and unbounded rewards, and showed the existence of the first passage optimal policies.…”

Section: Introductionmentioning

confidence: 99%

“…As is well known, as a generalization of the standard (infinite horizon) discounted DTMDPs as in [14,15], the first passage model of discounted DTMDPs has been widely studied in [5,8,16,24] for proving the existence of an optimal policy. In general, the main goal of this kind of study is to maximize (or minimize) the expected total discounted reward obtained (or cost paid) during a first passage time to some target set.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

First passage Markov decision processes with constraints and varying discount factors

Zou²,

Guo³

2015

Front. Math. China

Self Cite

View full text Add to dashboard Cite

This paper focuses on the constrained optimality problem (COP) of first passage discrete-time Markov decision processes (DTMDPs) in denumerable state and compact Borel action spaces with multi-constraints, state-dependent discount factors, and possibly unbounded costs. By means of the properties of a so-called occupation measure of a policy, we show that the constrained optimality problem is equivalent to an (infinite-dimensional) linear programming on the set of occupation measures with some constraints, and thus prove the existence of an optimal policy under suitable conditions. Furthermore, using the equivalence between the constrained optimality problem and the linear programming, we obtain an exact form of an optimal policy for the case of finite states and actions. Finally, as an example, a controlled queueing system is given to illustrate our results.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

First passage Markov decision processes with constraints and varying discount factors

Zou²,

Guo³

2015

Front. Math. China

Self Cite

View full text Add to dashboard Cite

show abstract

“…Close to the current paper work Guo et al (2012) is devoted the adaptive optimal control problem for discretetime, infinite horizon, stochastic control ''non-parametric'' systems where stochastic variables are i.i.d. random elements with unknown distribution.…”

Section: Introductionmentioning

confidence: 99%

Soil acidity adaptive control problem

Yakushev

Karelin

Bure

et al. 2014

Stoch Environ Res Risk Assess

View full text Add to dashboard Cite

Problem of soil acidity regularization is modeled as stochastic adaptive control problem with a linear difference equation of the dynamics of a field pH level. Stochastic component in the equation represents an individual time variability of soil acidity of an elementary section. We use Bayesian approach to determine a posteriori probability density function of the unknown parameters of the stochastic transition process. The KullbackLeibler information divergence is used as a measure of difference between true distribution and its estimation. Algorithm for the construction of an adaptive stabilizing control in such a linear control system is proposed in the paper. Numerical realization of the algorithm is represented for a problem of a field soil acidity control.

show abstract

“…Guo et al [15] consider a non-stationary dynamic where the performance index is a first passage type. Ye and Guo [28] consider the continuous-time case.…”

Section: Introductionmentioning

confidence: 99%