When the outcome is binary, psychologists often use nonlinear modeling strategies such as logit or probit. These strategies are often neither optimal nor justified when the objective is to estimate causal effects of experimental treatments. Researchers need to take extra steps to convert logit and probit coefficients into interpretable quantities, and when they do, these quantities often remain difficult to understand. Odds ratios, for instance, are described as obscure in many textbooks (e.g., Gelman & Hill, 2006, p. 83). I draw on econometric theory and established statistical findings to demonstrate that linear regression is generally the best strategy to estimate causal effects of treatments on binary outcomes. Linear regression coefficients are directly interpretable in terms of probabilities and, when interaction terms or fixed effects are included, linear regression is safer. I review the Neyman-Rubin Causal Model, which I use to prove analytically that linear regression yields unbiased estimates of treatment effects on binary outcomes. Then, I run simulations and analyze existing data on 24,191 students from 56 middle-schools (Paluck, Shepherd, & Aronow, 2019) to illustrate the effectiveness of linear regression. Based on these grounds, I recommend that psychologists use linear regression to estimate treatment effects on binary outcomes.

When the outcome is binary, psychologists often use nonlinear modeling strategies such as logit or probit. These strategies are often neither optimal nor justified when the objective is to estimate causal effects of experimental treatments. Researchers need to take extra steps to convert logit and probit coefficients into interpretable quantities, and when they do, these quantities often remain difficult to understand. Odds ratios, for instance, are described as obscure in many textbooks (e.g., Gelman & Hill, 2006, p. 83). I draw on econometric theory and established statistical findings to demonstrate that linear regression is generally the best strategy to estimate causal effects of treatments on binary outcomes. Linear regression coefficients are directly interpretable in terms of probabilities and, when interaction terms or fixed effects are included, linear regression is safer. I review the Neyman-Rubin Causal Model, which I use to prove analytically that linear regression yields unbiased estimates of treatment effects on binary outcomes.Then, I run simulations and analyze existing data on 24,191 students from 56 middle-schools (Paluck, Shepherd, & Aronow, 2016) to illustrate the effectiveness of linear regression. Based on these grounds, I recommend that psychologists use linear regression to estimate treatment effects on binary outcomes.

A pragmatist philosophy of psychological science offers to the direct replication debate concrete recommendations and novel benefits that are not discussed in Zwaan et al. This philosophy guides our work as field experimentalists interested in behavioral measurement. Furthermore, all psychologists can relate to its ultimate aim set out by William James: to study mental processes that provide explanations for why people behave as they do in the world.

Missing data is a common feature of experimental datasets. Standard methods used by psychology researchers to handle missingness rely on unrealistic assumptions, invalidate random assignment procedures, and bias estimates of effect sizes. In this tutorial, we describe different classes of missing data typically encountered in experimental datasets, and we discuss how each of them impacts researchers' causal inferences. We provide concrete guidelines for handling each class of missingness, focusing on two methods that make realistic assumptions: i) Inverse Probability Weighting (IPW) for mild instances of missingness, and ii) Double Sampling and Bounds for severe instances of missingness. After reviewing the reasons why these methods increase the accuracy of researchers' estimates of effect sizes, we provide lines of R code that researchers may use in their own analyses.

To what extent are television viewers affected by the behaviors and decisions they see modeled by characters in television soap operas? Collaborating with scriptwriters for three prime-time nationally-broadcast Spanish-language telenovelas, we embedded scenes about topics such as drunk driving or saving money at randomly assigned periods during the broadcast season. Outcomes were measured unobtrusively by aggregate city- and nation-wide time series, such as the number of Hispanic motorists arrested daily for drunk driving or the number of accounts opened in banks located in Hispanic neighborhoods. Results indicate that while two of the treatment effects are statistically significant, none are substantively large or long-lasting. Actions that could be taken during the immediate viewing session, like online searching, and those that were relatively more integrated into the telenovela storyline, specifically reducing cholesterol, were briefly affected, but not behaviors requiring sustained efforts, like opening a bank account or registering to vote.