Total Reward Variance in Discrete and Continuous Time Markov Chains

Sladký, Karel; Dijk, N.M. van

doi:10.1007/3-540-27679-3_40

Cited by 3 publications

(4 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is based on the mathematical framework of Markov chains with rewards (MCWR), introduced by Howard (1960) in the context of dynamic programming (see also, e.g., Benito 1982;Puterman 1994;Sladkỳ and van Dijk 2005). An individual moves among states according to a finite-state Markov chain.…”

Section: Markov Chains With Rewardsmentioning

confidence: 99%

Demography and the statistics of lifetime economic transfers under individual stochasticity

Caswell¹,

Kluge²

2015

DemRes

View full text Add to dashboard Cite

Section: Markov Chains With Rewardsmentioning

confidence: 99%

Demography and the statistics of lifetime economic transfers under individual stochasticity

Caswell¹,

Kluge²

2015

DemRes

View full text Add to dashboard Cite

“…Several authors in the widely scattered literature on Markov chains with rewards have addressed the variance of accumulated rewards. Sladky and van Dijk [45] , [46] have given results for discrete- and continuous-time chains with fixed rewards. Benito [42] provides variances for discrete chains with random rewards; my proof of Proposition 1 follows his approach.…”

Section: Discussionmentioning

confidence: 99%

Beyond R0: Demographic Models for Variability of Lifetime Reproductive Output

Caswell

2011

PLoS ONE

156

View full text Add to dashboard Cite

The net reproductive rate measures the expected lifetime reproductive output of an individual, and plays an important role in demography, ecology, evolution, and epidemiology. Well-established methods exist to calculate it from age- or stage-classified demographic data. As an expectation, provides no information on variability; empirical measurements of lifetime reproduction universally show high levels of variability, and often positive skewness among individuals. This is often interpreted as evidence of heterogeneity, and thus of an opportunity for natural selection. However, variability provides evidence of heterogeneity only if it exceeds the level of variability to be expected in a cohort of identical individuals all experiencing the same vital rates. Such comparisons require a way to calculate the statistics of lifetime reproduction from demographic data. Here, a new approach is presented, using the theory of Markov chains with rewards, obtaining all the moments of the distribution of lifetime reproduction. The approach applies to age- or stage-classified models, to constant, periodic, or stochastic environments, and to any kind of reproductive schedule. As examples, I analyze data from six empirical studies, of a variety of animal and plant taxa (nematodes, polychaetes, humans, and several species of perennial plants).

show abstract

“…The strategy to obtain the approximations is to exploit the fact that the evolution over time of total cost corresponds to the evolution of an aperiodic and recurrent Markov chain with an infinite number of states and a unique stationary distribution. In particular, we will use Markov reward process theory (Sladkỳ & van Dijk, 2005; van Dijk & Sladkỳ, 2006). Although Sladkỳ and van Dijk (2005) and van Dijk and Sladkỳ (2006) consider finite‐state Markov chains, their technique is also applicable to infinite‐state, aperiodic Markov chains with a unique stationary distribution, which is precisely the case for the evolution of K ( S ).…”

Section: Total Costmentioning

confidence: 99%

“…In particular, we will use Markov reward process theory (Sladkỳ & van Dijk, 2005; van Dijk & Sladkỳ, 2006). Although Sladkỳ and van Dijk (2005) and van Dijk and Sladkỳ (2006) consider finite‐state Markov chains, their technique is also applicable to infinite‐state, aperiodic Markov chains with a unique stationary distribution, which is precisely the case for the evolution of K ( S ).…”

Section: Total Costmentioning

confidence: 99%

On minimizing downside risk in make‐to‐stock, risk‐averse firms

Arreola‐Risa

2020

Naval Research Logistics

View full text Add to dashboard Cite

Consider a single‐product, make‐to‐stock, risk‐averse firm with stochastic demand and manufacturing capacity. Temporarily unfilled demands are back‐ordered. Total cost is equal to the sum of inventory‐holding and back‐ordering costs over a continuous‐time, finite planning horizon. In this article, we study the problem of finding the base‐stock level that minimizes total cost conditional value‐at‐risk (CVaR), or total cost CVaR for short. We demonstrate that total cost CVaR is both convex in the base‐stock level and increasing in risk aversion. We derive a closed‐form approximation of total cost which is asymptotically exact in the planning horizon length. The approximation leads to a relatively simple determination of optimal base‐stock levels which are reasonably accurate for practical applications. We make observations and identify findings regarding the impact on optimal base‐stock levels of changes in risk aversion, manufacturing capacity, inventory‐holding and back‐ordering costs, and planning horizon length. We provide a detailed description of how our research results were applied in a real‐world, supply‐chain design project.

show abstract

Total Reward Variance in Discrete and Continuous Time Markov Chains

Cited by 3 publications

References 5 publications

Demography and the statistics of lifetime economic transfers under individual stochasticity

Demography and the statistics of lifetime economic transfers under individual stochasticity

Beyond R0: Demographic Models for Variability of Lifetime Reproductive Output

On minimizing downside risk in make‐to‐stock, risk‐averse firms

Contact Info

Product

Resources

About