Panel Experiments and Dynamic Causal Effects: A Finite Population Perspective

Abstract

In panel experiments, we randomly expose multiple units to different treatments and measure their subsequent outcomes, sequentially repeating the procedure numerous times. Using the potential outcomes framework, we define finite population dynamic causal effects that capture the relative effectiveness of alternative treatment paths. For the leading example, known as the lag-p dynamic causal effects, we provide a nonparametric estimator that is unbiased over the randomization distribution. We then derive the finite population limiting distribution of our estimators as either the sample size or the duration of the experiment increases. Our approach provides a new technique for deriving finite population central limit theorems that exploits the underlying Martingale property of unbiased estimators. We further describe two methods for conducting inference on dynamic causal effects: a conservative test for weak null hypotheses of zero average causal effects using the limiting distribution and an exact randomization-based test for sharp null hypotheses. We also derive the finite population limiting distribution of commonly-used linear fixed effects estimators, showing that these estimators perform poorly in the presence of dynamic causal effects. We conclude with a simulation study and an empirical application in which we reanalyze a lab experiment on cooperation.

Double-Robust Two-Way-Fixed-Effects Regression For Panel Data

“…and , Bojinov et al [2020a], Roth and Sant'Anna [2021]). Similar to these papers, we establish properties of regression estimators under design assumptions.…”

Two-Way Fixed Effects and Differences-in-Differences with Heterogeneous Treatment Effects: A Survey

“…We restrict our attention to estimators relying on parallel trends assumptions, like TWFE regressions, but that do not restrict treatment effect heterogeneity between groups and over time, unlike TWFE regressions. This excludes papers that have assumed randomized treatment timing (see, e.g., Athey and Imbens, 2021;Roth and Sant'Anna, 2021) or sequential treatment randomization (see, e.g., Bojinov et al, 2020), rather than parallel trends. Intuitively, all the estimators below carefully choose valid control groups, to avoid making the "forbidden comparisons" that render TWFE estimators non-robust to heterogeneous treatment effects.…”

Estimating the causal effect of an intervention in a time series setting: the C-ARIMA approach

“…It means that the treatments assigned to the other units doesn't affect unit j's potential outcomes at the same time. This assumption, which is also known as Temporal Stable Unit Treatment Value Assumption or TSUTVA (Bojinov and Shephard, 2019;Bojinov et al, 2020), is the time series equivalent of the cross-sectional SUTVA (Rubin, 1974) and contributes to reduce the number of potential outcomes. In our empirical setting, the cookies selected for the permanent price discount differ on many characteristics, such as the shape, flavor and the ingredients, meaning that they appeal to different customers.…”

“…Having its roots in the context of randomized experiments, several methods have been developed to define and estimate causal effects under the RCM, including network data structures (VanderWeele, 2010;Forastiere et al, 2020;Noirjean et al, 2020), time series (Robins, 1986;Robins et al, 1999;Bojinov and Shephard, 2019) and panel data (Rambachan and Shephard, 2019;Bojinov et al, 2020). Unlike randomized experiments, however, in an observational study the researcher has no knowledge of, or no control on, the assignment mechanism, i.e., the process that determines the units receiving treatment and those under control.…”