Oracle-Efficient Online Learning and Auction Design

Abstract

We consider the design of computationally efficient online learning algorithms in an adversarial setting in which the learner has access to an offline optimization oracle. We present an algorithm called Generalized Follow-the-Perturbed-Leader and provide conditions under which it is oracle-efficient while achieving vanishing regret. Our algorithm generalizes the Follow-the-Perturbed-Leader (FTPL) approach of Kalai and Vempala [32] in which the action with the highest randomly perturbed historical performance is chosen on each round. FTPL is inefficient when the number of actions is exponential in the parameters of interest. Our algorithm creates more compact perturbations by augmenting the observed history with randomly generated synthetic history, and choosing the action with the (near) best performance on this augmented history. As we show, when certain structural properties hold, the augmented history is of polynomial size even when the learner's action space is exponential, yielding oracle-efficient learning. Our results make significant progress on an open problem raised by Hazan and Koren [27], who showed that oracle-efficient algorithms do not exist in general [26] and asked whether one can identify properties under which oracle-efficient online learning may be possible.Our second main contribution is the introduction of a new adversarial online auction-design framework for revenue maximization and the application of our oracle-efficient learning results to the adaptive design of auctions. In our framework, a seller repeatedly sells an item or set of items to a population of buyers by adaptively selecting auctions from a fixed target class. The goal of the seller is to leverage historical bid data to pick an auction on each iteration in such a way that the seller's overall revenue compares favorably with the revenue he would have obtained using the best auction from the class in hindsight. Since this is a specific case of adversarial online learning, we can apply our framework and provide new oracle-efficient learning results for: (1) Vickrey-Clarkes-Groves (VCG) auctions with bidder-specific reserves in single-parameter settings, (2) envy-free item pricing in multi-item auctions, and (3) the level auctions of Morgenstern and Roughgarden [37] for single-item settings. The last result leads to an approximation of the overall optimal Myerson auction when bidders' valuations are drawn according to a fast-mixing Markov process, extending prior work that only gave such guarantees for the i.i.d. setting.Finally, we derive various extensions, including: (1) oracle-efficient algorithms for the contextual learning setting in which the learner has access to side information (such as bidder demographics), (2) learning with approximate oracles such as those based on Maximal-in-Range algorithms, and (3) noregret bidding in simultaneous auctions, resolving an open problem of Daskalakis and Syrgkanis [13].

Minimizing Regret with Multiple Reserves

“…But in the MMR problem, the revenue is not a linear function of the reserves or of the valuations. 13 After the conference version of this article, Roughgarden and Wang (2016) and Dudik et al (2017) studied such "onlineto-offline reductions" in more general settings. One of their results gives conditions on an approximation algorithm under which it can be translated into an equally good online learning algorithm, and it includes our Theorem 4.1 as a special case.…”

Computing Stackelberg Equilibria of Large General-Sum Games

“…original game.Moreover, as we show in Theorem 1 there is a polynomial time algorithm for finding the Stackelberg equilibrium of such games when the leader can optimize a linear function over the actions of the follower, which is a similar condition to the ones used for computing Stackelberg equilibria in large zero-sum games (Ahmadinejad et al, 2016;Dudík et al, 2017;Kalai and Vempala, 2005).…”

“…We first note that when c e and C e are set to 0 for all e ∈ E, this game is zero-sum and can be efficiently solved when each player can compute its best-response to any choice of mixed strategy of the other player, i.e., optimize a linear function over the strategy space of the other player using existing results (Ahmadinejad et al, 2016;Dudík et al, 2017;Kalai and Vempala, 2005).…”

Oracle-efficient Online Learning and Auction Design

“…All our algorithms perform a single oracle call per round. a In the extended abstract [20], we presented the regret bound O (nm 2 √ T ) and running time O (T 2 + nmT ), corresponding to discretized level auctions with distinct thresholds (see Theorem 3.13). Here, we present the result that allows repetitions of threshold values (see Theorem 3.15).…”

“…b The regime of interest in this problem is s n. The extended abstract [20] contained a worse bound O (n 6 √ T ).…”