Markov Decision Problems Where Means Bound Variances

Arlotto, Alessandro; Gans, Noah; Steele, J. Michael

doi:10.1287/opre.2014.1281

Cited by 13 publications

(8 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For a particular class of problems, relevant in the study of Markov decision processes where means bound variances (Arlotto et al 2014), Proposition 4 automatically implies full exploration of every post-decision state, as stated in the following corollary.…”

Section: Asymptotic Analysis Of Offline Kgmentioning

confidence: 96%

Bayesian Exploration for Approximate Dynamic Programming

Ryzhov

Mes²,

Powell

et al. 2019

Operations Research

View full text Add to dashboard Cite

07-425.R4 Authors are encouraged to submit new papers to INFORMS journals by means of a style file template, which includes the journal title. However, use of a template does not certify that the paper has been accepted for publication in the named journal. INFORMS journal templates are for the exclusive purpose of submitting to an INFORMS journal and should not be used to distribute the papers in print or online or to submit the papers to another publication.

show abstract

Section: Asymptotic Analysis Of Offline Kgmentioning

confidence: 96%

Bayesian Exploration for Approximate Dynamic Programming

Ryzhov

Mes²,

Powell

et al. 2019

Operations Research

View full text Add to dashboard Cite

show abstract

“…This condition does not fully escape the Saint Petersburg Paradox, but it does give one some practical guidance. Moreover, it was found in Arlotto et al (2014) that such closeness in probability holds automatically for a large class of Markov decision problems.…”

Section: Weighing Costs With More Than Meansmentioning

confidence: 99%

A Central Limit Theorem for Costs in Bulinskaya’s Inventory Management Problem When Deliveries Face Delays

Arlotto

Steele

2016

Methodol Comput Appl Probab

Self Cite

View full text Add to dashboard Cite

It is common in inventory theory to consider policies that minimize the expected cost of ordering and holding goods or materials. Nevertheless, the realized cost is a random variable, and, as the Saint Petersburg Paradox reminds us, the expected value does not always capture the full economic reality of a decision problem. Here we take the classic inventory model of Bulinskaya (1964), and, by proving an appropriate central limit theorem, we show in a reasonably rich (and practical) sense that the mean-optimal policies are economically appropriate. The motivation and the tools are applicable to a large class of Markov decision problems. (2010): Primary: 60C05, 90B05; Secondary: 60F05, 60J05, 90C39, 90C40. Mathematics Subject Classification

show abstract

“…Papastavrou et al [12]), only little is known about the limiting distribution of the optimal number of size-focused knapsack selections. From Arlotto et al [2] one has a suggestive upper bound on the variance, but at this point one does not know for sure that this bound gives the principle term of the variance for large n.…”

Section: Observations Connections and Problemsmentioning

confidence: 99%

Quickest online selection of an increasing subsequence of specified size

Arlotto

Mossel

Steele

2016

Random Struct. Alg.

Self Cite

View full text Add to dashboard Cite

Abstract. Given a sequence of independent random variables with a common continuous distribution, we consider the online decision problem where one seeks to minimize the expected value of the time that is needed to complete the selection of a monotone increasing subsequence of a prespecified length n. This problem is dual to some online decision problems that have been considered earlier, and this dual problem has some notable advantages. In particular, the recursions and equations of optimality lead with relative ease to asymptotic formulas for mean and variance of the minimal selection time.Mathematics Subject Classification (2010): Primary: 60C05, 90C40; Secondary: 60G40, 90C27, 90C39Key Words: Increasing subsequence problem, online selection, sequential selection, time-focused decision problem, dynamic programming, Markov decision problem. Increasing Subsequences and Time-Focused SelectionIf X 1 , X 2 , . . . is a sequence of independent random variables with a common continuous distribution F , then Here we consider a new kind of decision problem where one seeks to select as quickly as possible an increasing subsequence of a prespecified length n. More precisely, at time i, when the decision maker is first presented with X i , a decision

show abstract

Markov Decision Problems Where Means Bound Variances

Cited by 13 publications

References 42 publications

Bayesian Exploration for Approximate Dynamic Programming

Bayesian Exploration for Approximate Dynamic Programming

A Central Limit Theorem for Costs in Bulinskaya’s Inventory Management Problem When Deliveries Face Delays

Quickest online selection of an increasing subsequence of specified size

Contact Info

Product

Resources

About