Learning, Experimentation, and Information Design

Abstract

for detailed comments. 2 Equilibrium Interactions 2.1 Strategic Bandits Strategic bandit models are game-theoretic versions of standard bandit models. While the standard "multi-armed bandit" describes a hypothetical experiment in which a player faces several slot machines ("one-armed bandits") with potentially different expected payouts, a strategic bandit involves several players facing (usually, identical) copies of the same slot machine. Players want to stick with the slot machine if and only if the best payout rate makes it worth their time, and learn not only from their own outcomes but also from their neighbors. Equilibrium strategies are not characterized by simple cutoffs in terms of the common belief. As a result, solutions to strategic bandits are only known for a limited class of distributions, involving two states of the world only. In Bolton and Harris (1999, BH), the observation process (of payoffs) follows a Brownian motion, whose drift depends on the state. In Keller, Rady and Cripps (2005, KRC), it follows a simple Poisson process, with positive lump-sums ("breakthroughs") occurring at random (exponentially distributed) times if and only if the arm is good. Keller and Rady (2015, KR15) solve the polar opposite case in which costly lump-sums ("breakdowns") occur at random times if and only if the arm is bad. Keller and Rady (2010, KR10) consider the case in which breakthroughs need not be conclusive. These models share in common, in addition to the binary state framework, their focus on symmetric Markov perfect equilibria (MPE). 1 Throughout, players are Bayesian. Given that they observe all actions and outcomes, they share a common belief about the state, which serves as the state variable. They are also impatient and share a common discount rate. BH and KR10 are the most ambitious models and offer no closed-form solutions for the equilibrium. Remarkably, however, they are able to prove uniqueness and tease out not only their structure, but their dependence on parameters. While it is BH that first develops both the ideas (including the concepts of free-riding and encouragement effects) as well as the methods used throughout this literature, the most interesting insights can already be gleaned from the simple exponential bandits in KRC and KR15. What makes these two models tractable is that these models can be viewed as deterministic: unless a breakdown or breakthrough ("news") occurs, the (conditional) posterior belief follows a known path. If news arrives, the game is over, since if it is commonly known that the state is good (or bad), informational externalities cease to matter, and each player knows the strictly dominant action to take. Let us consider here a simplified version combining the good and bad news models. The state is ω ∈ {G, B}. Each player i = 1,. .. , I controls the variable u i t ∈ [0, 1], which is the fraction allocated to the risky arm at time t ≥ 0 (the complementary fraction being allocated to the safe arm). The horizon is infinite. This leads to a total realized payoff of ...

Optimal Dynamic Information Acquisition

“…• Poisson learning relates to Wald's problem (Wald (1947), Che & Mierendorff (2016)). Poisson bandit is used as a building block for strategic experimentation models (see a survey by Hörner & Skrzypacz (2016)). My example considers a simplest stationary Poisson stopping strategy that stochastically reveals the true state.…”

Bandits in the lab

“…Bergemann and Välimäki (1996, 2000) considered strategic experimentation in buyer‐seller settings. Hörner and Skrzypacz (2017) gave a survey of this literature.…”

Learning from Others' Outcomes

“…Equilibrium paths with this feature are arguably implausible, because the inferences players must draw based on their samples are quite complex: for instance, in the example there are times where players take the risky action if they observe a small or large number of successes and take the safe action if they observe an intermediate number of successes. 24 To capture this idea, I say that an equilibrium path is simple if at each point in time observing success is always unambiguously good news or bad news: for each t, X (t) ( ) is either increasing or decreasing in . 25 Proposition 8 If > for some innovation-optimal state , then there does not exist a simple asymptotically e¢ cient path.…”