The equivalence of the Delta method and the cluster-robust variance estimator for the analysis of clustered randomized experiments

Abstract

It often happens that the same problem presents itself to different communities and the solutions proposed or adopted by those communities are different. We take the case of the variance estimation of the population average treatment effect in cluster-randomized experiments. The econometrics literature promotes the cluster-robust variance estimator [Athey and Imbens, 2017], which can be dated back to the study of linear regression with clustered residuals [Liang and Zeger, 1986]. The A/B testing or online experimentation literature promotes the delta method [Kohavi et al., 2010, Deng et al., 2017, 2018, which tackles the variance estimation of the ATE estimator directly using large sample theory. The two methods are seemly different as the former begins with a regression setting at the individual unit level and the latter is semi-parametric with only i.i.d. assumptions on the clusters. Both methods are widely used in practice. It begs the question for their connection and comparison. In this paper we prove they are equivalent and in the canonical implementation they should give exactly the same result.