Between Stochastic and Adversarial Online Convex Optimization: Improved Regret Bounds via Smoothness

Abstract

Stochastic and adversarial data are two widely studied settings in online learning. But many optimization tasks are neither i.i.d. nor fully adversarial, which makes it of fundamental interest to get a better theoretical understanding of the world between these extremes. In this work we establish novel regret bounds for online convex optimization in a setting that interpolates between stochastic i.i.d. and fully adversarial losses. By exploiting smoothness of the expected losses, these bounds replace a dependence on the maximum gradient length by the variance of the gradients, which was previously known only for linear losses. In addition, they weaken the i.i.d. assumption by allowing adversarially poisoned rounds or shifts in the data distribution. To accomplish this goal, we introduce two key quantities associated with the loss sequence, that we call the cumulative stochastic variance and the adversarial variation. Our upper bounds are attained by instances of optimistic follow the regularized leader, and we design adaptive learning rates that automatically adapt to the cumulative stochastic variance and adversarial variation. In the fully i.i.d. case, our bounds match the rates one would expect from results in stochastic acceleration, and in the fully adversarial case they gracefully deteriorate to match the minimax regret. We further provide lower bounds showing that our regret upper bounds are tight for all intermediate regimes for the cumulative stochastic variance and the adversarial variation.

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion

“…Moreover, our analysis assumes perfect tuning of constants (e.g., D, T, K) for simplicity. In practice, we would prefer to adapt to unknown parameters, motivating new applications and problems for adaptive online learning, which is already an area of active current investigation (see, e.g., Orabona & Pál, 2015;Hoeven et al, 2018;Cutkosky & Orabona, 2018;Cutkosky, 2019;Mhammedi & Koolen, 2020;Chen et al, 2021;Sachs et al, 2022;Wang et al, 2022). It is our hope that some of this expertise can be applied in the non-convex setting as well.…”