Universal Regression with Adversarial Responses

Abstract

We provide algorithms for regression with adversarial responses under large classes of non-i.i.d. instance sequences, on general separable metric spaces, with provably minimal assumptions. We also give characterizations of learnability in this regression context. We consider universal consistency which asks for strong consistency of a learner without restrictions on the value responses. Our analysis shows that such objective is achievable for a significantly larger class of instance sequences than stationary processes, and unveils a fundamental dichotomy between value spaces: whether finite-horizon mean-estimation is achievable or not. We further provide optimistically universal learning rules, i.e., such that if they fail to achieve universal consistency, any other algorithm will fail as well. For unbounded losses, we propose a mild integrability condition under which there exist algorithms for adversarial regression under large classes of non-i.i.d. instance sequences. In addition, our analysis also provides a learning rule for meanestimation in general metric spaces that is consistent under adversarial responses without any moment conditions on the sequence, a result of independent interest.

Non-stationary Contextual Bandits and Universal Learning

“…This implies that adapting algorithms for specific context processes is necessary to ensure universal learning. This is the first example of such a phenomenon for online learning, for which previously considered settings always admitted optimistically universal learning rules, including realizable (noiseless) supervised learning [2,15,17], arbitrarily noisy (potentially adversarial rewards) supervised learning [19,20], and stationary contextual bandits [1]. Intuitively, we show that personalization and generalization are incompatible for non-stationary contextual bandits.…”

“…This comes in stark contrast with all the learning frameworks that have been studied in the universal learning literature. Namely, for the noiseless full-feedback [2,17], noisy/adversarial full-feedback [20] and stationary partial-feedback [1] learning frameworks, analysis showed that there always existed an optimistically universal learning rule. Precisely, the optimistically universal learning rule for stationary contextual bandits in finite action spaces provided by [1] combined two strategies:…”

“…For more general noisy data generating processes [19,20] gave complete characterizations and showed that universal learning can be achieved not only for noisy data but arbitrarily dependent values Y on the contexts X, possibly even adversarial to the learner's predictions. Specifically, [20] showed that under mild assumptions on the value space-including totally-boundedmetric spaces-optimistically universal learning with noisy values is possible on the exact same class of processes as for noiseless values. Hence, learning with arbitrary or adversarial responses comes at no generality expense for the full-feedback setting.…”

“…In this section, we ask the question whether there exists an optimistically universal learning rule for finite action spaces. In fact, in all the frameworks considered for universal learning-noiseless [17] or noisy/adversarial responses [20] in the full-feedback setting and stationary partial-feedback responses [1]-analysis showed that optimistically universal learning always existed. However, the learning rule provided by [1] for stationary rewards under C 2 processes heavily relies on the assumption that the rewards are stationary in order to make good estimates of the performance of different learning strategies.…”