On the Impossibility of Statistically Improving Empirical Optimization: A Second-Order Stochastic Dominance Perspective

Abstract

When the underlying probability distribution in a stochastic optimization is observed only through data, various data-driven formulations have been studied to obtain approximate optimal solutions. We show that no such formulations can, in a sense, theoretically improve the statistical quality of the solution obtained from empirical optimization. We argue this by proving that the first-order behavior of the optimality gap against the oracle best solution, which includes both the bias and variance, for any data-driven solution is second-order stochastically dominated by empirical optimization, as long as suitable smoothness holds with respect to the underlying distribution. We demonstrate this impossibility of improvement in a range of examples including regularized optimization, distributionally robust optimization, parametric optimization and Bayesian generalizations. We also discuss the connections of our results to semiparametric statistical inference and other perspectives in the data-driven optimization literature.

Statistical Analysis of Wasserstein Distributionally Robust Estimators

“…Under regularity conditions, the optimality gap obtained by the empirical risk minimization solution, technically defined as the difference R P * , θ erm n − inf θ∈Θ R(P * , θ), is asymptotically optimal as the sample size increases in the second-order convex sense compared to a wide range of regularizations including the DRO-type formulations [53]. While this optimality property is remarkable, it is important to keep in perspective the assumptions imposed therein.…”

“…While this optimality property is remarkable, it is important to keep in perspective the assumptions imposed therein. For example, conditions such as the existence of a unique optimizer, twice differentiability at the optimum, and a fixed dimensionality environment appear key in the development of the asymptotic optimality result in [53]. Furthermore, in more complex decision making tasks, the notion of optimality gap may require refinements.…”

How Big Should Your Data Really Be? Data-Driven Newsvendor: Learning One Sample at a Time

“…We highlight here that there has been significant work on data-driven policies. We discuss more these in the literature review and refer to Lam (2021) for a very recent overview of various subfamilies of policies considered in the literature. We emphasize here that when searching for optimal policies, we consider all possible mappings from data to decisions, and do not restrict attention to a subfamily of policies.…”