Sharp bounds on the variance in randomized experiments

Aronow, Peter M.; Green, Donald P.; Lee, Donald K. K.

doi:10.1214/13-aos1200

Cited by 65 publications

(71 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We then leverage these results to bound an R 2 -like measure of treatment effect variation explained by covariates. In the Supplementary Material we also show that we can use these results to obtain sharp bounds on the variance of Neyman (1923)'s estimate of the Average Treatment Effect, extending previous work by Heckman et al (1997) and Aronow et al (2014).…”

Section: Idiosyncratic Treatment Effect Variation For Ittsupporting

confidence: 73%

“…For more technical discussion, see Ding (2014), Aronow et al (2014, and Middleton and Aronow (2015); for regularity conditions of the finite population central limit theorems, see Hájek (1960) and Lehmann (1998).…”

Section: Randomization-based Estimatormentioning

confidence: 99%

“…Neyman (1923) initially proposed a lower bound for var( τ ) under the assumption of a constant treatment effect, S τ τ = 0. More recently, Aronow et al (2014) proposed to bound S τ τ via Fréchet-Hoeffding bounds rather than to assume S τ τ = 0 (cf. Heckman et al, 1997).…”

Section: Appendix B3 Variance Of the Average Treatment Effect Estimatementioning

confidence: 99%

See 2 more Smart Citations

Decomposing Treatment Effect Variation

Ding

Feller

Miratrix³

2018

Journal of the American Statistical Association

View full text Add to dashboard Cite

Understanding and characterizing treatment effect variation in randomized experiments has become essential for going beyond the "black box" of the average treatment effect. Nonetheless, traditional statistical approaches often ignore or assume away such variation. In the context of a randomized experiment, this paper proposes a framework for decomposing overall treatment effect variation into a systematic component that is explained by observed covariates, and a remaining idiosyncratic component. Our framework is fully randomization-based, with estimates of treatment effect variation that are fully justified by the randomization itself. Our framework can also account for noncompliance, which is an important practical complication. We make several key contributions. First, we show that randomization-based estimates of systematic variation are very similar in form to estimates from fully-interacted linear regression and two stage least squares. Second, we use these estimators to develop an omnibus test for systematic treatment effect variation, both with and without noncompliance. Third, we propose an R 2 -like measure of treatment effect variation explained by covariates and, when applicable, noncompliance. Finally, we assess these methods via simulation studies and apply them to the Head Start Impact Study, a large-scale randomized experiment.

show abstract

Section: Idiosyncratic Treatment Effect Variation For Ittsupporting

confidence: 73%

Section: Randomization-based Estimatormentioning

confidence: 99%

See 1 more Smart Citation

Decomposing Treatment Effect Variation

Ding

Feller

Miratrix³

2018

Journal of the American Statistical Association

View full text Add to dashboard Cite

show abstract

“…rerandomization with x = ∅ or a = ∞), expression (26) provides asymptotically exact confidence intervals. In general, these confidence intervals can be conservative and can be improved (Aronow et al, 2014;Fogarty, 2018;Ding et al, 2019). We focus on expressions (22) and (26) for their simplicity, and divide this section into two subsections in parallel with Section 5.…”

Section: Optimal Adjustment Based On the Estimated Distributionmentioning

confidence: 99%

Rerandomization and Regression Adjustment

Ding

2019

Journal of the Royal Statistical Society Series B: Statistical Methodology

View full text Add to dashboard Cite

Summary Randomization is a basis for the statistical inference of treatment effects without strong assumptions on the outcome‐generating process. Appropriately using covariates further yields more precise estimators in randomized experiments. R. A. Fisher suggested blocking on discrete covariates in the design stage or conducting analysis of covariance in the analysis stage. We can embed blocking in a wider class of experimental design called rerandomization, and extend the classical analysis of covariance to more general regression adjustment. Rerandomization trumps complete randomization in the design stage, and regression adjustment trumps the simple difference‐in‐means estimator in the analysis stage. It is then intuitive to use both rerandomization and regression adjustment. Under the randomization inference framework, we establish a unified theory allowing the designer and analyser to have access to different sets of covariates. We find that asymptotically, for any given estimator with or without regression adjustment, rerandomization never hurts either the sampling precision or the estimated precision, and, for any given design with or without rerandomization, our regression‐adjusted estimator never hurts the estimated precision. Therefore, combining rerandomization and regression adjustment yields better coverage properties and thus improves statistical inference. To quantify these statements theoretically, we discuss optimal regression‐adjusted estimators in terms of the sampling precision and the estimated precision, and then measure the additional gains of the designer and the analyser. We finally suggest the use of rerandomization in the design and regression adjustment in the analysis followed by the Huber–White robust standard error.

show abstract

“…See, for example, Neyman (1923Neyman ( ,1990, Aronow, Green, and Lee (2014), Ding (2017), and Abadie, Athey, Imbens, and Wooldridge (2017).…”

Section: The Causal Bootstrap For Average Treatment Effectsmentioning

confidence: 99%

A Causal Bootstrap

Imbens¹,

Menzel²

2018

View full text Add to dashboard Cite

The bootstrap, introduced by Efron (1982), has become a very popular method for estimating variances and constructing confidence intervals. A key insight is that one can approximate the properties of estimators by using the empirical distribution function of the sample as an approximation for the true distribution function. This approach views the uncertainty in the estimator as coming exclusively from sampling uncertainty. We argue that for causal estimands the uncertainty arises entirely, or partially, from a different source, corresponding to the stochastic nature of the treatment received. We develop a bootstrap procedure that accounts for this uncertainty, and compare its properties to that of the classical bootstrap.JEL Classification: C1, C14, C18

show abstract

Sharp bounds on the variance in randomized experiments

Cited by 65 publications

References 18 publications

Decomposing Treatment Effect Variation

Decomposing Treatment Effect Variation

Rerandomization and Regression Adjustment

A Causal Bootstrap

Contact Info

Product

Resources

About