Phantom Counterfactuals

Abstract

Researchers often seek to identify the effects of a treatment on a sequence of behaviors, such as whether citizens register to vote and whether they then cast ballots. I show that average treatment effects (ATEs) are only identified until the first behavior (registering to vote) that affects the set of possible subsequent actions (voting). When one action changes the set of possible subsequent actions, it creates ‘phantom counterfactuals,’ or undefined potential outcomes, which render ATEs unidentified. I show that applied theory allows researchers to diagnose phantom counterfactuals, which helps to recognize unidentified ATEs and focus instead on other estimands that are identified. I illustrate this approach using a stylized model of crime reporting, showing how different theories generate different sets of identified estimands while holding constant an experimental design. I thereby establish the necessity of applied theory for causal identification in empirical research with sequential behavioral outcomes.

Can Close Election Regression Discontinuity Designs Identify Effects of Winning Politician Characteristics?

“…Third, this article illustrates the need to understand why candidates end up in close elections, and thus complements recent work emphasizing the theoretical implications of empirical models (e.g. Ashworth, Berry, and Bueno de Mesquita 2021; Bueno de Mesquita and Tyson 2020; Eggers 2017, Slough Forthcoming) and the large econometric literature documenting how sample selection generates bias (e.g., Heckman 1979).…”

Justice Bork, or, be careful which supreme court nominees you choose to fight

“…This type of analysis is informative in scenarios where there is an extensive knowledge of the variables that could have affected the outcome in question (Fearon 1991; Goertz and Mahoney 2012). Within‐case counterfactual analysis can be useful when examining potential outcomes related to a particular choice (Greenstein 1998), though problems can arise when multiple individual choices are linked together and are responsible for a single outcome (Goertz and Mahoney 2012; Slough 2023; Tetlock and Lebow 2001; Weingast 1996). Rather, qualitative counterfactual analysis should adhere to the minimum rewrite rule , which states that counterfactual analyses should change as little of the historical record as possible.…”

Eclipse: How Darkness Shapes Violence in Africa

“…However, it would be premature to interpret Table A9-3 as support for the psychological explanation. 5 5 I do not conduct mediation analysis or compare the effects with and without violent events, as the mediator, violence, is endogenous, and thus, the core assumption, sequential ignorability, is not satisfied (Bansak 2020;Imai, Keele, and Tingley 2010;Keele and Stevenson 2021;Slough 2023;Zhou and Yamamoto 2023). It is implausible to assume that violence only depends on solar eclipses and the remaining variance is (conditionally) ignorable.…”