Learning to Bid Optimally and Efficiently in Adversarial First-price Auctions

Abstract

First-price auctions have very recently swept the online advertising industry, replacing secondprice auctions as the predominant auction mechanism on many platforms. This shift has brought forth important challenges for a bidder: how should one bid in a first-price auction, where unlike in second-price auctions, it is no longer optimal to bid one's private value truthfully and hard to know the others' bidding behaviors? In this paper, we take an online learning angle and address the fundamental problem of learning to bid in repeated first-price auctions, where both the bidder's private valuations and other bidders' bids can be arbitrary. We develop the first minimax optimal online bidding algorithm that achieves an O( √ T ) regret when competing with the set of all Lipschitz bidding policies, a strong oracle that contains a rich set of bidding strategies. This novel algorithm is built on the insight that the presence of a good expert can be leveraged to improve performance, as well as an original hierarchical expert-chaining structure, both of which could be of independent interest in online learning. Further, by exploiting the product structure that exists in the problem, we modify this algorithm-in its vanilla form statistically optimal but computationally infeasible-to a computationally efficient and space efficient algorithm that also retains the same O( √ T ) minimax optimal regret guarantee. Additionally, through an impossibility result, we highlight that one is unlikely to compete this favorably with a stronger oracle (than the considered Lipschitz bidding policies). Finally, we test our algorithm on three real-world first-price auction datasets obtained from Verizon Media and demonstrate our algorithm's superior performance compared to several existing bidding algorithms.

No-regret Learning in Repeated First-Price Auctions with Budget Constraints

“…Repeated first-price auctions without budgets Two notable works concerning repeated firstprice auctions are Han et al [2020b] and Han et al [2020a]. In Han et al [2020b], they introduce a new problem called monotone group contextual bandits and then obtain an O( √ T ln 2 T )-regret algorithm for repeated first-price auctions without budget constraints under stationary settings.…”

“…In Han et al [2020b], they introduce a new problem called monotone group contextual bandits and then obtain an O( √ T ln 2 T )-regret algorithm for repeated first-price auctions without budget constraints under stationary settings. In Han et al [2020a], they concentrate on an adversarial setting and develop a mini-max optimal online bidding algorithm with O( √ T ln T ) regret against all Lipschitz bidding strategies. A crucial difference is that in the present paper, budgets are involved thus the bandit algorithm by Han et al [2020b] is not suitable for our needs.…”

Nash Convergence of Mean-Based Learning Algorithms in First Price Auctions

“…Online learning in auctions Many works on online learning in auctions are about "learning to bid", focusing on how to design no-regret algorithms for a bidder to bid in various formats of repeated auctions, including first price auctions (Balseiro et al, 2019;Han et al, 2020), second price auctions (Iyer et al, 2014;Weed et al, 2016), and more general auctions (Feng et al, 2018;Karaca et al, 2020). These works take the perspective of a single bidder, without considering the interaction among multiple bidders all of whom learn to bid at the same time.…”

Beyond Bandit Feedback in Online Multiclass Classification

“…Other variants include sleeping experts (Cortes et al, 2019), switching experts (Arora et al, 2019), and adaptive adversaries (Feng and Loh, 2018). Some works use feedback graphs to bound the regret in auctions (Cesa-Bianchi et al, 2017;Han et al, 2020). In the stochastic setting, regret bounds for Thompson sampling and UCB have been analyzed by Tossou et al (2017); Liu et al (2018); Lykouris et al (2020).…”