Test &amp; Roll: Profit-Maximizing A/B Tests

Feit, Elea McDonnell; Berman, Ron

doi:10.1287/mksc.2019.1194

Cited by 40 publications

(7 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For most providers, we suspect that the marginal differences in the cost and complexity of deploying different messages tested in our megastudy would be vanishingly small. However, we hope that the transparency with which we've reported our results will allow policy makers interested in deploying our findings to make the right cost-benefit tradeoffs for their organizations (29)(30)(31)(32).…”

Section: Discussionmentioning

confidence: 97%

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Milkman

Gandhi

Patel

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

136

View full text Add to dashboard Cite

Encouraging vaccination is a pressing policy problem. To assess whether text-based reminders can encourage pharmacy vaccination and what kinds of messages work best, we conducted a megastudy. We randomly assigned 689,693 Walmart pharmacy patients to receive one of 22 different text reminders using a variety of different behavioral science principles to nudge flu vaccination or to a business-as-usual control condition that received no messages. We found that the reminder texts that we tested increased pharmacy vaccination rates by an average of 2.0 percentage points, or 6.8%, over a 3-mo follow-up period. The most-effective messages reminded patients that a flu shot was waiting for them and delivered reminders on multiple days. The top-performing intervention included two texts delivered 3 d apart and communicated to patients that a vaccine was “waiting for you.” Neither experts nor lay people anticipated that this would be the best-performing treatment, underscoring the value of simultaneously testing many different nudges in a highly powered megastudy.

show abstract

Section: Discussionmentioning

confidence: 97%

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Milkman

Gandhi

Patel

et al. 2022

Proc. Natl. Acad. Sci. U.S.A.

Self Cite

136

View full text Add to dashboard Cite

show abstract

“…Ideally, the decision to experiment and to integrate experimental findings into profit maximization can help the manager to make use of the results for subsequent marketing decisions. Feit and Berman (2019) provide a recent example of such a framing. Digital ad sellers may require techniques to validate their experimentation platforms. An increasing number of platforms offer experimentation services to help measure ad effects.…”

Section: Digital Advertising Effect Measurementmentioning

confidence: 99%

Section: Future Research Opportunitiesmentioning

confidence: 99%

Inefficiencies in Digital Advertising Markets

Gordon¹,

Jerath

Katona

et al. 2020

Journal of Marketing

112

View full text Add to dashboard Cite

Digital advertising markets are growing and attracting increased scrutiny. This article explores four market inefficiencies that remain poorly understood: ad effect measurement, frictions between and within advertising channel members, ad blocking, and ad fraud. Although these topics are not unique to digital advertising, each manifests in unique ways in markets for digital ads. The authors identify relevant findings in the academic literature, recent developments in practice, and promising topics for future research.

show abstract

“…An alternative to searching for naturally occurring variation is to undertake field experiments that have been designed to evaluate the long-term effects of retention campaigns. Field experiments have been used extensively in the domains of customer acquisition and response to marketing actions such as display advertising, website design, and catalog mailings (Feit and Berman 2019).…”

Section: Opportunities and Future Research Directions In Harnessing Cmentioning

confidence: 99%

Capturing Marketing Information to Fuel Growth

Du¹,

Netzer²,

Schweidel³

et al. 2020

Journal of Marketing

View full text Add to dashboard Cite

Marketing is the functional area primarily responsible for driving the organic growth of a firm. In the age of digital marketing and big data, marketers are inundated with increasingly rich data from an ever-expanding array of sources. Such data may help marketers generate insights about customers and competitors. One fundamental question remains: How can marketers wrestle massive flows of existing and nascent data resources into coherent, effective growth strategies? Against such a backdrop, the Marketing Science Institute has made “capturing information to fuel growth” a top research priority. We begin by discussing the streetlight effect—an overreliance on readily available data due to ease of measurement and application—as contributing to the disconnect between marketing data growth and firm growth. We then use the customer equity framework to structure our discussion of six areas where we see substantial under-tapped opportunities—incorporating social network and biometric data in customer acquisition, trend and competitive interaction data in customer development, and unstructured and causal data in customer retention. We highlight challenges that obstruct firms from realizing such data-driven growth opportunities and how future research may help overcome those challenges.

show abstract

Test & Roll: Profit-Maximizing A/B Tests

Abstract: We reframe A/B testing as a decision problem with the goal of maximizing profit and derive the optimal sample size formula.

Cited by 40 publications

References 23 publications

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

A 680,000-person megastudy of nudges to encourage vaccination in pharmacies

Inefficiencies in Digital Advertising Markets

Capturing Marketing Information to Fuel Growth

Contact Info

Product

Resources

About