Bias of the regression estimator for experiments using clustered random assignment

Middleton, Joel A.

doi:10.1016/j.spl.2008.03.008

Cited by 18 publications

(18 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This stochasticity causes many complications to inference—for example, the mean‐difference estimator is often biased in cluster‐randomised designs, even when all units' responses can be measured (Middleton, ; Middleton & Aronow, ). Consequently, there is not a straightforward analog to our results that will hold for cluster‐randomised designs; indeed, our results will not even hold for the simple two‐cluster example mentioned in the previous paragraph.…”

Section: Sampling Properties Of the Estimator Of The Average Treatmenmentioning

confidence: 99%

“…Because we require that N 0 and N 1 are fixed, our results will hold for cluster‐randomised designs if the number of units within each cluster is the same across clusters, but this is rarely the case. As discussed in Middleton () and Middleton and Aronow (), cluster sizes and the covariance between treatment group sizes and treatment effects are important quantities for deriving inferential properties of ATE estimators in cluster‐randomised designs, and likely, these quantities would be similarly important in deriving analogous results when there is a sampling stage after treatment assignment at the cluster level. We leave this for future work.…”

Section: Sampling Properties Of the Estimator Of The Average Treatmenmentioning

confidence: 99%

See 1 more Smart Citation

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Branson

Dasgupta

2019

Int Statistical Rev

View full text Add to dashboard Cite

Summary We establish the inferential properties of the mean‐difference estimator for the average treatment effect in randomised experiments where each unit in a population is randomised to one of two treatments and then units within treatment groups are randomly sampled. The properties of this estimator are well understood in the experimental design scenario where first units are randomly sampled and then treatment is randomly assigned but not for the aforementioned scenario where the sampling and treatment assignment stages are reversed. We find that the inferential properties of the mean‐difference estimator under this experimental design scenario are identical to those under the more common sample‐first‐randomise‐second design. This finding will bring some clarifications about sampling‐based randomised designs for causal inference, particularly for settings where there is a finite super‐population. Finally, we explore to what extent pre‐treatment measurements can be used to improve upon the mean‐difference estimator for this randomise‐first‐sample‐second design. Unfortunately, we find that pre‐treatment measurements are often unhelpful in improving the precision of average treatment effect estimators under this design, unless a large number of pre‐treatment measurements that are highly associative with the post‐treatment measurements can be obtained. We confirm these results using a simulation study based on a real experiment in nanomaterials.

show abstract

Section: Sampling Properties Of the Estimator Of The Average Treatmenmentioning

confidence: 99%

Section: Sampling Properties Of the Estimator Of The Average Treatmenmentioning

confidence: 99%

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Branson

Dasgupta

2019

Int Statistical Rev

View full text Add to dashboard Cite

show abstract

“…As the setup of the natural experiment is based on one treated cluster and four untreated clusters, the number of clusters may be too small to generate unbiased standard errors for the DID (Middleton ; Middleton and Aronow 2015), and the smallest possible p‐value of randomization inference would equal 1/5 = 0.2. As a consequence, Figure only displays point estimates and simulated distributions without references to statistical significance .…”

Section: Empirical Analysismentioning

confidence: 99%

When Do Campaign Effects Persist for Years? Evidence from a Natural Experiment

Jäger

2019

American J Political Sci

View full text Add to dashboard Cite

A unique revote allows for a natural experiment to evaluate whether campaign effects can last for nearly a decade: A right-wing conservative party missed the 5% threshold in a German state by a mere vote in 2007, but the Constitutional Court ordered a revote in a single precinct over potential election fraud. After a one-sided campaign focusing on law and order, the party's vote share increased more than sixfold. By comparing the precinct with its direct surroundings, the study shows that the revote campaign had long-lasting effects on vote choice and broader security-sensitive behavior. Residents in the revote precinct installed more warning signs on their property to deter burglars. They were not more supportive of right-wing attitudes but were more likely to believe that election fraud reoccurred. Based on habitual-voting and social-norm theories, the study suggests that persuasion could be durable if candidates provide an unchallenged interpretation of political events.

show abstract

“…where the cluster-level random effects c i 's are IID with mean zero and variance δ 2 , the ε ij 's are IID with mean zero and variance σ 2 , and the c i 's and ε ij 's are independent and follows joint Normality. See Middleton (2008) and Schochet (2013) for a discussion of the properties and especially the disadvantages of regression analysis for estimating the average causal effects in clustered randomized trials.…”

Section: Test Statistics Based On Ranksmentioning

confidence: 99%

Rank tests in unmatched clustered randomized trials applied to a study of teacher training

Ding¹,

Keele²

2018

Ann. Appl. Stat.

View full text Add to dashboard Cite

In the Teacher and Leader Performance Evaluation Systems study, schools were randomly assigned to receive new measures of teacher and principal performance. One outcome in the study, measured at the teacher level, was truncated at zero, and displayed a long tail. Rank-based statistics are one natural method to apply to such outcomes, since inferences will be robust and exact, and we can avoid assumptions about the model that generated the data. We investigate four different possible rank statistics that vary in the form of weighting applied to clusters. Each test statistic has the correct level but may vary in terms of the power to detect departures from the null. We conduct a simulation for power comparing to linear mixed models with Normal, t, and Cauchy errors. We obtain a point estimate and construct confidence intervals by applying the Tobit model of effects, which assumes that treatment increases the outcome by a constant amount but only if the response under control would be positive. We also develop a formal randomization-based method for testing the appropriateness of the Tobit model of effects. In the data from the study, we find no evidence against the Tobit model of effects.

show abstract

Bias of the regression estimator for experiments using clustered random assignment

Cited by 18 publications

References 16 publications

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

Sampling‐based Randomised Designs for Causal Inference under the Potential Outcomes Framework

When Do Campaign Effects Persist for Years? Evidence from a Natural Experiment

Rank tests in unmatched clustered randomized trials applied to a study of teacher training

Contact Info

Product

Resources

About