Customer Acquisition via Display Advertising Using Multi-Armed Bandit Experiments

Abstract

Firms using online advertising regularly run experiments with multiple versions of their ads since they are uncertain about which ones are most effective. During a campaign, firms try to adapt to intermediate results of their tests, optimizing what they earn while learning about their ads. Yet how should they decide what percentage of impressions to allocate to each ad? This paper answers that question, resolving the well-known "learn-and-earn" trade-off using multi-armed bandit (MAB) methods. The online advertiser's MAB problem, however, contains particular challenges, such as a hierarchical structure (ads within a website), attributes of actions (creative elements of an ad), and batched decisions (millions of impressions at a time), that are not fully accommodated by existing MAB methods. Our approach captures how the impact of observable ad attributes on ad effectiveness differs by website in unobserved ways, and our policy generates allocations of impressions that can be used in practice. We implemented this policy in a live field experiment delivering over 750 million ad impressions in an online display campaign with a large retail bank. Over the course of two months, our policy achieved an 8% improvement in the customer acquisition rate, relative to a control policy, without any additional costs to the bank. Beyond the actual experiment, we performed counterfactual simulations to evaluate a range of alternative model specifications and allocation rules in MAB policies. Finally, we show that customer acquisition would decrease by about 10% if the firm were to optimize clickthrough rates instead of conversion directly, a finding that has implications for understanding the marketing funnel. This paper answers that question, resolving the well-known "learn-and-earn" trade-off using multi-armed bandit (MAB) methods. The online advertiser's MAB problem, however, contains particular challenges, such as a hierarchical structure (ads within a website), attributes of actions (creative elements of an ad), and batched decisions (millions of impressions at a time), that are not fully accommodated by existing MAB methods. Our approach captures how the impact of observable ad attributes on ad effectiveness differs by website in unobserved ways, and our policy generates allocations of impressions that can be used in practice.We implemented this policy in a live field experiment delivering over 700 million ad impressions in an online display campaign with a large retail bank. Over the course of two months, our policy achieved an 8% improvement in the customer acquisition rate, relative to a control policy, without any additional costs to the bank. Beyond the actual experiment, we performed counterfactual simulations to evaluate a range of alternative model specifications and allocation rules in MAB policies. Finally, we show that customer acquisition would decrease about 10% if the firm were to optimize click through rates instead of conversion directly, a finding that has implications for understanding the marketi...

Personalizing persuasive technologies: Explicit and implicit personalization using persuasion profiles

“…It then becomes of interest to study policies which prescribe actions that maximize the reward over all interactions (e.g., max P t ¼ T t ¼ 1 R t jA t ; X t ). There is a broad literature on the topic (see, e.g., Berry and Fristedt, 1985;Audibert et al, 2009;Scott, 2010), and within the marketing literature researchers are already approaching the personalization problem as a contextual bandit problem (Hauser et al, 2009(Hauser et al, , 2014Schwartz et al, 2013). Appreciation of the inherent uncertainty in the coupling between user, message content, and observed behavior by exploring different policies is, in our view, a key next step for the development of personalized persuasive systems.…”

Editorial—Marketing Science and Big Data

“…Another combines methods from cloud computing, machine learning, and text mining to demonstrate how online social platforms such as Twitter can be used for sales forecasting (Liu et al 2016). A fifth paper that will appear in a subsequent issue, uses methods from multi-armed bandit problems to identify the banner-advertising characteristics that are most likely to appeal to consumers (Schwartz et al 2016).…”

Optimal Product Design by Sequential Experiments in High Dimensions

“…Another stream proposes to select a query that minimizes regret or cost, when making decisions under strict uncertainty (e.g., Wang and Boutilier 2003). An independent yet related methodology is the multi-armed bandit (MAB), which simultaneously considers maximum performance and minimum regret (Schwartz et al 2016). It is widely used in online field experiments for digital content optimization (e.g., Brezzi and Lai 2002, Scott 2010, Schwartz et al 2016 to balance between profits during the experimental phase and the optimal outcome.…”