Online Nonparametric Regression with General Loss Functions

Rakhlin, Alexander; Sridharan, Karthik

doi:10.48550/arxiv.1501.06598

Cited by 7 publications

(8 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A full proof is in Appendix C. Combining Theorem 3 with [HRS21, Theorem 3.2], we have a complete characterization of the statistical rates of smooth online learning. As a final remark, we note that our proof applied to the results of [RS15] immediately extends to nonconstructively show that the dependence of the minimax value on T for squared loss are the expected "fast rates" from [RST17]. Unfortunately, our efficient algorithms below do not provably achieve these rates; resolving this disparity is an interesting future direction with applications to the study of contextual bandits, as described in Appendix A.…”

Section: Minimax Valuementioning

confidence: 82%

“…This characterization was extended to the case of constrained adversaries (including the special case of smoothed adversaries) in [RST11]. Several subsequent papers have established further refined regret bounds [BDR21,RS15]. The profusion of publications relating to algorithmic questions about online learning is too large to enumerate here, but notable relevant work includes [HK16], which provides lower bounds on oracle-efficiency and [RSS12] which introduces a general framework for constructing algorithms.…”

Section: B Related Workmentioning

confidence: 99%

See 1 more Smart Citation

Smoothed Online Learning is as Easy as Statistical Learning

Block¹,

Dagan²,

Golowich³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Much of modern learning theory has been split between two regimes: the classical offline setting, where data arrive independently, and the online setting, where data arrive adversarially. While the former model is often both computationally and statistically tractable, the latter requires no distributional assumptions. In an attempt to achieve the best of both worlds, previous work proposed the smooth online setting where each sample is drawn from an adversarially chosen distribution, which is smooth, i.e., it has a bounded density with respect to a fixed dominating measure. Existing results for the smooth setting were known only for binary-valued function classes and were computationally expensive in general; in this paper, we fill these lacunae. In particular, we provide tight bounds on the minimax regret of learning a nonparametric function class, with nearly optimal dependence on both the horizon and smoothness parameters. Furthermore, we provide the first oracle-efficient, no-regret algorithms in this setting. In particular, we propose an oracle-efficient improper algorithm whose regret achieves optimal dependence on the horizon and a proper algorithm requiring only a single oracle call per round whose regret has the optimal horizon dependence in the classification setting and is sublinear in general. Both algorithms have exponentially worse dependence on the smoothness parameter of the adversary than the minimax rate. We then prove a lower bound on the oracle complexity of any proper learning algorithm, which matches the oracle-efficient upper bounds up to a polynomial factor, thus demonstrating the existence of a statistical-computational gap in smooth online learning. Finally, we apply our results to the contextual bandit setting to show that if a function class is learnable in the classical setting, then there is an oracle-efficient, no-regret algorithm for contextual bandits in the case that contexts arrive in a smooth manner.

show abstract

Section: Minimax Valuementioning

confidence: 82%

Section: B Related Workmentioning

confidence: 99%

Smoothed Online Learning is as Easy as Statistical Learning

Block¹,

Dagan²,

Golowich³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Here, the gap between the PAC-misspecified and worst-case is very easy to demonstrate (take Y i = 1{X i ≤ θ} with θ ∈ [0, 1] -the 1D-barrier -which cannot be predicted, Γ n = ∞, in the worst case, but is easy in the iid case). For increasingly more general losses, [24,21,22] show that regret can be sharply characterized by the metric-entropy type quantites (sequential Rademacher complexity). However, for the log-loss it turns out that the entropic characterization is not possible, cf.…”

Section: Motivation and Literaturementioning

confidence: 99%

Sequential prediction under log-loss and misspecification

Feder¹,

Polyanskiy²

2021

Preprint

View full text Add to dashboard Cite

We consider the question of sequential prediction under the logloss in terms of cumulative regret. Namely, given a hypothesis class of distributions, learner sequentially predicts the (distribution of the) next letter in sequence and its performance is compared to the baseline of the best constant predictor from the hypothesis class. The well-specified case corresponds to an additional assumption that the data-generating distribution belongs to the hypothesis class as well.Here we present results in the more general misspecified case. Due to special properties of the log-loss, the same problem arises in the context of competitive-optimality in density estimation, and model selection. For the d-dimensional Gaussian location hypothesis class, we show that cumulative regrets in the well-specified and misspecified cases asymptotically coincide. In other words, we provide an o(1) characterization of the distribution-free (or PAC) regret in this casethe first such result as far as we know. We recall that the worst-case (or individual-sequence) regret in this case is larger by an additive constant d 2 + o(1). Surprisingly, neither the traditional Bayesian estimators, nor the Shtarkov's normalized maximum likelihood achieve the PAC regret and our estimator requires special "robustification" against heavy-tailed data. In addition, we show two general results for misspecified regret: the existence and uniqueness of the optimal estimator, and the bound sandwiching the misspecified regret between well-specified regrets with (asymptotically) close hypotheses classes.

show abstract

“…As for the applications of the chaining idea in online learning, the log loss was studied in [OH99,CBL01], and [RS14,RST17] considered the square loss. For general loss functions, the online nonparametric regression problem was studied in [GG15,RS15], and [CBGGG17] considered general contextual bandits with full or censored feedbacks. However, a key distinguishing factor between the previous works and ours is that the loss (or reward) function in first-price auctions is not continuous, resulting in a potentially large performance gap among the children of a single internal node in the chain and rendering the previous arguments inapplicable.…”

Section: Related Workmentioning

confidence: 99%

Learning to Bid Optimally and Efficiently in Adversarial First-price Auctions

Han,

Zhou,

Flores

et al. 2020

Preprint

View full text Add to dashboard Cite

First-price auctions have very recently swept the online advertising industry, replacing secondprice auctions as the predominant auction mechanism on many platforms. This shift has brought forth important challenges for a bidder: how should one bid in a first-price auction, where unlike in second-price auctions, it is no longer optimal to bid one's private value truthfully and hard to know the others' bidding behaviors? In this paper, we take an online learning angle and address the fundamental problem of learning to bid in repeated first-price auctions, where both the bidder's private valuations and other bidders' bids can be arbitrary. We develop the first minimax optimal online bidding algorithm that achieves an O( √ T ) regret when competing with the set of all Lipschitz bidding policies, a strong oracle that contains a rich set of bidding strategies. This novel algorithm is built on the insight that the presence of a good expert can be leveraged to improve performance, as well as an original hierarchical expert-chaining structure, both of which could be of independent interest in online learning. Further, by exploiting the product structure that exists in the problem, we modify this algorithm-in its vanilla form statistically optimal but computationally infeasible-to a computationally efficient and space efficient algorithm that also retains the same O( √ T ) minimax optimal regret guarantee. Additionally, through an impossibility result, we highlight that one is unlikely to compete this favorably with a stronger oracle (than the considered Lipschitz bidding policies). Finally, we test our algorithm on three real-world first-price auction datasets obtained from Verizon Media and demonstrate our algorithm's superior performance compared to several existing bidding algorithms.

show abstract

Online Nonparametric Regression with General Loss Functions

Cited by 7 publications

References 20 publications

Smoothed Online Learning is as Easy as Statistical Learning

Smoothed Online Learning is as Easy as Statistical Learning

Sequential prediction under log-loss and misspecification

Learning to Bid Optimally and Efficiently in Adversarial First-price Auctions

Contact Info

Product

Resources

About