An Aggregation Scheme for Increased Power

Abstract

A novel aggregation scheme increases power in randomized controlled trials and quasi-experiments when the intervention possesses a robust and well-articulated theory of change. Longitudinal data analyzing interventions often include multiple observations on individuals, some of which may be more likely to manifest a treatment effect than others. An intervention's theory of change provides guidance as to which of those observations are best situated to exhibit that treatment effect. Our power-maximizing w eighting for r epeated-measurements with d elayed-effects scheme, PWRD aggregation, converts the theory of change into a test statistic with improved Pitman efficiency, delivering tests with greater statistical power. We illustrate this method on an IES-funded cluster randomized trial testing the efficacy of a reading intervention designed to assist early elementary students at risk of falling behind their peers. The salient theory of change holds program benefits to be delayed and non-uniform, experienced after a student's performance stalls. This intervention is not found to have an effect, but the PWRD technique's effect on power is found to be comparable to that of a doubling of (cluster-level) sample size.