Smoothed Online Learning is as Easy as Statistical Learning

Abstract

Much of modern learning theory has been split between two regimes: the classical offline setting, where data arrive independently, and the online setting, where data arrive adversarially. While the former model is often both computationally and statistically tractable, the latter requires no distributional assumptions. In an attempt to achieve the best of both worlds, previous work proposed the smooth online setting where each sample is drawn from an adversarially chosen distribution, which is smooth, i.e., it has a bounded density with respect to a fixed dominating measure. Existing results for the smooth setting were known only for binary-valued function classes and were computationally expensive in general; in this paper, we fill these lacunae. In particular, we provide tight bounds on the minimax regret of learning a nonparametric function class, with nearly optimal dependence on both the horizon and smoothness parameters. Furthermore, we provide the first oracle-efficient, no-regret algorithms in this setting. In particular, we propose an oracle-efficient improper algorithm whose regret achieves optimal dependence on the horizon and a proper algorithm requiring only a single oracle call per round whose regret has the optimal horizon dependence in the classification setting and is sublinear in general. Both algorithms have exponentially worse dependence on the smoothness parameter of the adversary than the minimax rate. We then prove a lower bound on the oracle complexity of any proper learning algorithm, which matches the oracle-efficient upper bounds up to a polynomial factor, thus demonstrating the existence of a statistical-computational gap in smooth online learning. Finally, we apply our results to the contextual bandit setting to show that if a function class is learnable in the classical setting, then there is an oracle-efficient, no-regret algorithm for contextual bandits in the case that contexts arrive in a smooth manner.

Efficient and Near-Optimal Smoothed Online Learning for Generalized Linear Functions

“…In this section, we provide basic definitions and setup the learning problem. We begin by defining a smooth distribution, as in Block et al [2022], Haghtalab et al [2021]: Definition 1. Let µ be a probability measure on a measurable space X.…”

“…To circumvent the pessimism of the sequential setting, recent works [Rakhlin et al, 2011, Haghtalab et al, 2020, Block et al, 2022, Haghtalab et al, 2022 have studied the smoothed sequential learning paradigm, where the adversary is constrained to choose x t at random from any probability distribution p t with density at most 1/σ with respect to a known measure µ. The most current of these results point to a striking statistical computational gap: whereas there exist algorithms which attain regret that scales with T log(/σ), computationally efficient algorithms can only hope for poly(T /σ) regret in general, even against a realizable adversary [Haghtalab et al, 2022, Theorem 5.2].…”

“…Finally, we present a complementary approach based on the perceptron algorithm which is robust to adversarial corruptions of the labels y t , and enjoys a polynomial regret in a "directional smoothness" parameter which interpolates between the log(1/σ)-guarantees attained above in the realizable setting, and the poly(1/σ) bounds from prior work. We emphasize that, though we adopt the smoothed online learning setting of Rakhlin et al [2011], Haghtalab et al [2021], Block et al [2022], we use entirely different techniques involving Ville's inequality [Ville, 1939], geometric measure theory, and convex geometry. Moreover, in none of these works was the question of adapting to realizability explored; thus, we provide the first regret bounds that are logarithmic in both the horizon and the smoothness parameter.…”

“…Recently, Block et al [2022], Haghtalab et al [2022] generalized Haghtalab et al [2021] to allow for continuous labels and, more importantly, provided oracle-efficient algorithms for achieving vanishing regret in the smoothed setting. These papers also showed that the dependence on σ in the regret bounds of their oracle-efficient algorithms, which was polynomial, could not in general be reduced to the logarithmic dependence achievable by the inefficient algorithms, thereby exposing a statistical-computational gap.…”

“…These papers also showed that the dependence on σ in the regret bounds of their oracle-efficient algorithms, which was polynomial, could not in general be reduced to the logarithmic dependence achievable by the inefficient algorithms, thereby exposing a statistical-computational gap. Unlike other recent works such as Block et al [2022], Haghtalab et al [2022], we do not use the coupling approach [Haghtalab et al, 2021] to prove our regret bounds.…”