Prediction with Corrupted Expert Advice

Abstract

We revisit the fundamental problem of prediction with expert advice, in a setting where the environment is benign and generates losses stochastically, but the feedback observed by the learner is subject to a moderate adversarial corruption. We prove that a variant of the classical Multiplicative Weights algorithm with decreasing step sizes achieves constant regret in this setting and performs optimally in a wide range of environments, regardless of the magnitude of the injected corruption. Our results reveal a surprising disparity between the often comparable Follow the Regularized Leader (FTRL) and Online Mirror Descent (OMD) frameworks: we show that for experts in the corrupted stochastic regime, the regret performance of OMD is in fact strictly inferior to that of FTRL.

Stochastic Graphical Bandits with Adversarial Corruptions

“…This gap was partially bridged by Zimmert and Seldin (2019), who proved that online mirror descent with Tsallis-INF regularizer can achieve the optimal O(K ln T + C) bound in expectation, provided that the optimal arm is unique. Recently, the adversariallycorrupted stochastic reward model is extended to prediction with expert advice (Amir et al 2020), assortment optimization (Chen, Krishnamurthy, and Wang 2019), Gaussian bandits (Bogunovic, Krause, and Scarlett 2020), linear bandits (Kapoor, Patel, and Kar 2019;Li, Lou, and Shan 2019), and reinforcement learning (Lykouris et al 2019). Instead of studying the budget-bounded corruption setting, several papers focus on the scenario where the rewards are corrupted with a fixed probability (Altschuler, Brunel, and Malek 2019;Kapoor, Patel, and Kar 2019;Guan et al 2020).…”

“…Thus, a natural question arises: Is there a bandits model that lies between the stochastic and adversary worlds and admits regret guarantees only slightly worsen than the logarithmic regret bound? In fact, this question has been answered affirmatively in the context of MAB (Seldin and Slivkins 2014;Lykouris, Mirrokni, and Paes Leme 2018;Zimmert and Seldin 2019;Gupta, Koren, and Talwar 2019) and PEA (Amir et al 2020), but still remains open for GB.…”

“…Besides extending BARBAR to graphical bandits, we also make improvements and refinements of BARBAR, including a different estimation of mean reward and a smaller constant for epoch length, which help to reduce the leading constant factor in the regret bound of BARBAR by more than 500 times and achieve better empirical performance. Finally, for stochastic PEA with adversarial corruptions, a special case of our setting with α = 1, the existing algorithm only enjoys an expected regret bound and requires the optimal expert to be unique (Amir et al 2020). By contrast, our algorithm achieves a high probability regret bound and can handle scenarios with multiple optimal experts.…”

Prediction against a limited adversary

“…The fact that the adversary can only interfere on the outcome of the prediction of one expert is the main difference between our framework and the classical prediction with expert advice problems in [2,10,11,15,18], and also the bandit problems with corruption such as [19] and [25] where the regret bounds provided depend on the corruption. However, unlike [3] and [21], the adversary can optimally control the level of corruption at each round and therefore the level of corruption might be unbounded.…”

“…Similar to [3,8,19,21,25], in this paper, we bridge the two cases by considering an adversary who cannot freely choose the outcomes. In our framework, the gains of the experts are drawn from a fixed distribution.…”