A Bayesian Risk Approach to Data-driven Stochastic Optimization: Formulations and Asymptotics

Wu, Di; Zhu, Helin; Zhou, Enlu

doi:10.1137/16m1101933

Cited by 29 publications

(35 citation statements)

References 38 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They estimate the parameter uncertainty using a Bayesian posterior distribution, and impose a risk functional on the objective function with respect to the posterior distribution. Later Wu et al [2018] show the consistency and robustness against parameter uncertainty of the BRO framework.…”

Section: Introductionmentioning

confidence: 91%

“…Then the problem turns into finding the optimal policy that minimizes the expected total cost under the most adversarial distribution in the ambiguity set. However, their ambiguity set is constructed only on apriori probabilistic information of the unknown parameters and is not updated even when more data can be observed later; moreover, distributionally robust approach can sometimes result in overly conservative solutions (Wu et al [2018]). In view of the conservativeness of the distributionally robust approach, Zhou and Xie [2015] propose a Bayesian risk optimization (BRO) framework that reformulates the (single-stage) stochastic optimization problem with parameter uncertainty.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Bayesian Risk Markov Decision Processes

Lin¹,

Ren²,

Zhou³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

We consider Markov Decision Processes (MDPs) where distributional parameters, such as transition probabilities, are unknown and estimated from data. The popular distributionally robust approach to addressing the parameter uncertainty can sometimes be overly conservative. In this paper, we propose a Bayesian risk approach to MDPs with parameter uncertainty, where a risk functional is applied in nested form to the expected discounted total cost with respect to the Bayesian posterior distributions of the unknown parameters in each time stage. The proposed approach provides more flexibility of risk attitudes towards parameter uncertainty and takes into account the availability of data in future time stages. For the finite-horizon MDPs, we show the dynamic programming equations can be solved efficiently with an upper confidence bound (UCB) based adaptive sampling algorithm. For the infinite-horizon MDPs, we propose a risk-adjusted Bellman operator and show the proposed operator is a contraction mapping that leads to the optimal value function to the Bayesian risk formulation. We demonstrate the empirical performance of our proposed algorithms in the finite-horizon case on an inventory control problem and a path planning problem.Preprint. Under review.

show abstract

Section: Introductionmentioning

confidence: 91%

Section: Introductionmentioning

confidence: 99%

Bayesian Risk Markov Decision Processes

Lin¹,

Ren²,

Zhou³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recently, Wu et al [57] proposed a data-driven Bayesian optimization approach. The authors apply a risk functional towards the expected value E θ k Q(x, ξ) .…”

Section: Bayesian Approachmentioning

confidence: 99%

“…This Bayesian risk optimization framework was already proposed before [60], whereas tailored solution strategies were first presented in [61]. For the case of an infinite number of observations, Wu et al [57] prove several consistency and asymptotic results. This is the main theoretical difference to our work, where we establish bounds for a finite number of observations; cf.…”

Section: Bayesian Approachmentioning

confidence: 99%

Data-driven stochastic optimization for distributional ambiguity with integrated confidence region

Rebennack

2022

J Glob Optim

View full text Add to dashboard Cite

We discuss stochastic optimization problems under distributional ambiguity. The distributional uncertainty is captured by considering an entire family of distributions. Because we assume the existence of data, we can consider confidence regions for the different estimators of the parameters of the distributions. Based on the definition of an appropriate estimator in the interior of the resulting confidence region, we propose a new data-driven stochastic optimization problem. This new approach applies the idea of a-posteriori Bayesian methods to the confidence region. We are able to prove that the expected value, over all observations and all possible distributions, of the optimal objective function of the proposed stochastic optimization problem is bounded by a constant. This constant is small for a sufficiently large i.i.d. sample size and depends on the chosen confidence level and the size of the confidence region. We demonstrate the utility of the new optimization approach on a Newsvendor and a reliability problem.

show abstract

“…The paper [25] studies the connection between robust optimization and variance regularization by developing an expansion of the robust objective, which we discuss in Section 3, but is an in-sample analysis. We also note the paper [44] which studies the asymptotic properties of stochastic optimization problems with risk-sensitive objectives.…”

Section: Introductionmentioning

confidence: 99%

Calibration of Distributionally Robust Empirical Optimization Models

2017

View full text Add to dashboard Cite

In this paper, we study the out-of-sample properties of robust empirical optimization and develop a theory for data-driven calibration of the "robustness parameter" for worst-case maximization problems with concave reward functions. Building on the intuition that robust optimization reduces the sensitivity of the expected reward to errors in the model by controlling the spread of the reward distribution, we show that the first-order benefit of "little bit of robustness" is a significant reduction in the variance of the out-of-sample reward while the corresponding impact on the mean is almost an order of magnitude smaller. One implication is that a substantial reduction in the variance of the out-of-sample reward (i.e. sensitivity of the expected reward to model misspecification) is possible at little cost if the robustness parameter is properly calibrated. To this end, we introduce the notion of a robust mean-variance frontier to select the robustness parameter and show that it can be approximated using resampling methods like the bootstrap. Our examples also show that "open loop" calibration methods (e.g. selecting a 90% confidence level regardless of the data and objective function) can lead to solutions that are very conservative out-of-sample.

show abstract

A Bayesian Risk Approach to Data-driven Stochastic Optimization: Formulations and Asymptotics

Cited by 29 publications

References 38 publications

Bayesian Risk Markov Decision Processes

Bayesian Risk Markov Decision Processes

Data-driven stochastic optimization for distributional ambiguity with integrated confidence region

Calibration of Distributionally Robust Empirical Optimization Models

Contact Info

Product

Resources

About