Bridging Finite and Super Population Causal Inference

Ding, Peng; Li, Xinran; Miratrix, Luke

doi:10.1515/jci-2016-0027

Cited by 28 publications

(20 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An important note is that in some cases, these sampling schemes are chosen for convenience and that the generalizability of the experiment to the population will depend upon the assumptions made in them being true. The sampling model may also be considered to serve as a conservative approach to finite sample inference (see Ding et al, 2017).…”

Section: Pashley and Miratrixmentioning

confidence: 99%

Insights on Variance Estimation for Blocked and Matched Pairs Designs

Pashley¹,

Miratrix

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

Evaluating blocked randomized experiments from a potential outcomes perspective has two primary branches of work. The first focuses on larger blocks, with multiple treatment and control units in each block. The second focuses on matched pairs, with a single treatment and control unit in each block. These literatures not only provide different estimators for the standard errors of the estimated average impact, but they are also built on different sets of assumptions. Neither literature handles cases with blocks of varying size that contain singleton treatment or control units, a case which can occur in a variety of contexts, such as with different forms of matching or poststratification. In this article, we reconcile the literatures by carefully examining the performance of variance estimators under several different frameworks. We then use these insights to derive novel variance estimators for experiments containing blocks of different sizes.

show abstract

Section: Pashley and Miratrixmentioning

confidence: 99%

Insights on Variance Estimation for Blocked and Matched Pairs Designs

Pashley¹,

Miratrix

2020

Journal of Educational and Behavioral Statistics

Self Cite

View full text Add to dashboard Cite

show abstract

“…First, and most importantly: Almost all power results implicitly assume S 2 τ = 0 (Lachin, 1981;Cohen, 1992;Lerman, 1996;Wittes, 2002); indeed, because S 2 τ cannot directly be estimated, it is common to assume S 2 τ = 0 and use the variance estimator Vn . Furthermore, if a super-population is assumed, S 2 τ is not involved in the asymptotic variance of the mean-difference estimator (Imbens & Rubin 2015, Chapter 6;Ding et al 2017). However, under a finite-population framework, even when one uses the estimator Vn , S 2 τ is still involved in the true variance of τ , and thus is also involved in the power of a completely randomized experiment.…”

Section: Inference and Powermentioning

confidence: 99%

Power and Sample Size Calculations for Rerandomized Experiments

Branson¹,

Li²,

Ding³

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Power is an important aspect of experimental design, because it allows researchers to understand the chance of detecting causal effects if they exist. It is common to specify a desired level of power, and then compute the sample size necessary to obtain that level of power; thus, power calculations help determine how experiments are conducted in practice. Power and sample size calculations are readily available for completely randomized experiments; however, there can be many benefits to using other experimental designs. For example, in recent years it has been established that rerandomized designs, where subjects are randomized until a prespecified level of covariate balance is obtained, increase the precision of causal effect estimators. This work establishes the statistical power of rerandomized treatment-control experiments, thereby allowing for sample size calculators. Our theoretical results also clarify how power and sample size are affected by treatment effect heterogeneity, a quantity that is often ignored in power analyses. Via simulation, we confirm our theoretical results and find that rerandomization can lead to substantial sample size reductions; e.g., in many realistic scenarios, rerandomization can lead to a 25% or even 50% reduction in sample size for a fixed level of power, compared to complete randomization. Power and sample size calculators based on our results are in the R package rerandPower on CRAN.

show abstract

“…Consequently,

v_{\overset{τ}{true}} = s_{0}^{2} false/ n_{0} + s_{1}^{2} false/ n_{1}

is a natural Neymanian‐style estimator of

Var false(\overset{τ}{true} false)

and

Var false(\overset{τ}{true} false)

, which, as seen in Section , has an upward bias of

S_{τ}^{2} false/ N

unless strict additivity holds. This estimator—originating in Neyman ()—is by far the most common estimator for the variance of the ATE in randomised experiments (Rubin, ; Imbens, ; Miratrix et al , ; Imbens & Rubin, ; Ding et al , ), and thus, it is reassuring that it can be used for

Var false(\overset{τ}{true} false)

and

Var false(\overset{τ}{true} false)

, that is, it can be used under Design 1 or Design 2.…”

Section: Sampling Properties Of the Estimator Of The Average Treatmenmentioning

confidence: 99%

“…This finding also clarifies some discussions in the causal inference literature that consider a super‐population framework. When discussing causal inference under a super‐population framework, many works implicitly assume that N >> n or N = ∞ , and thus, the third term in

Var ((), \overset{τ}{true}), S_{τ}^{2} false/ N

, is often ignored (Imbens ; Imbens & Rubin, , Chapter 6; Ding et al ., ; Sekhon & Shem‐Tov, ). As we will see in Section , this term is ignored in the estimation of

Var false(\overset{τ}{true} false)

Var false(\overset{τ}{true} false)

, because the observed data do not provide any information about

S_{τ}^{2}

, because none of the individual τ i are ever observed.…”

Section: Sampling Properties Of the Estimator Of The Average Treatmenmentioning

confidence: 99%

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Branson

Dasgupta

2019

Int Statistical Rev

View full text Add to dashboard Cite

Summary We establish the inferential properties of the mean‐difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean‐difference estimator under this experimental design scenario are identical to those under the more common sample‐first‐randomise‐second design. This finding will bring some clarifications about sampling‐based randomised designs for causal inference, particularly for settings where there is a finite super‐population. Finally, we explore to what extent pre‐treatment measurements can be used to improve upon the mean‐difference estimator for this randomise‐first‐sample‐second design. Unfortunately, we find that pre‐treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre‐treatment measurements that are highly associative with the post‐treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

show abstract

Bridging Finite and Super Population Causal Inference

Cited by 28 publications

References 26 publications

Insights on Variance Estimation for Blocked and Matched Pairs Designs

Insights on Variance Estimation for Blocked and Matched Pairs Designs

Power and Sample Size Calculations for Rerandomized Experiments

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Contact Info

Product

Resources

About