Online Stochastic Optimization with Wasserstein Based Non-stationarity

Abstract

We consider a general online stochastic optimization problem with multiple budget constraints over a horizon of finite time periods. In each time period, a reward function and multiple cost functions are revealed, and the decision maker needs to specify an action from a convex and compact action set to collect the reward and consume the budget. Each cost function corresponds to the consumption of one budget. In each time period, the reward function and the cost functions are drawn from an unknown distribution, which is non-stationary across time. The objective of the decision maker is to maximize the cumulative reward subject to the budget constraints. This formulation captures a wide range of applications including online linear programming and network revenue management, among others. In this paper, we consider two settings:(i) a data-driven setting where the true distribution is unknown but a prior estimate (possibly inaccurate) is available; (ii) an uninformative setting where the true distribution is completely unknown. We propose a unified Wasserstein-distance based measure to quantify the inaccuracy of the prior estimate in setting (i) and the non-stationarity of the system in setting (ii). We show that the proposed measure leads to a necessary and sufficient condition for the attainability of a sublinear regret in both settings. For setting (i), we propose an informative gradient descent algorithm. The algorithm takes a primal-dual perspective and it integrates the prior information of the underlying distributions into an online gradient descent procedure in the dual space. The algorithm also naturally extends to the uninformative setting (ii). Under both settings, we show the corresponding algorithm achieves a regret of optimal order. In numerical experiments, we demonstrate how the proposed algorithms can be naturally integrated with the re-solving technique to further boost the empirical performance.

Online Primal-Dual Algorithms For Stochastic Resource Allocation Problems

“…As shown in Lemma 1, both the optimal gap and the constraint violation between the LP problem (12) and SOCP problem (16) are O( √ n). Then, putting Theorem 1, Theorem 2, and Lemma 1 together, we can obtain that the expected optimality gap and constraint violation compared to the optimal solution of the SOCP problem (16) in Theorem 3.…”

“…In the existing articles that study the uncertain online optimization, the uncertainty is modeled by the worst-case scenario value, expectation, regret, or a linear combination of the above (see [5,4,16,22,17]). These modeling methods are mainly aimed at the uncertainty in the objective function, while almost no chance constraint or conditional expectation constraint is considered in the existing studies.…”

“…The optimal objective values of the ISOCP problem (10) and SOCP problem ( 16) are referred to as RISOCP n and RSOCP n , respectively. Then, we establish the optimality gap between the LP problem (12) and SOCP problem (16), and the constraint violation in the following lemma.…”

“…Lemma 1. Under Assumption 1, we have the following results on the SOCP problem (16) and LP problem (12).…”

“…Moreover, inequality RISOCP n ≤ RSOCP n holds due to the fact that the SOCP problem (16) is the linear relaxation of the ISOCP problem (10). Thus, Algorithm 1 achieves O( √ n) regret and constraint violation compared to the optimal solution of the ISOCP problem (10) as stated in Theorem 4.…”