Statistical analysis and handling of missing data in cluster randomized trials: a systematic review

Abstract

BackgroundCluster randomized trials (CRTs) randomize participants in groups, rather than as individuals and are key tools used to assess interventions in health research where treatment contamination is likely or if individual randomization is not feasible. Two potential major pitfalls exist regarding CRTs, namely handling missing data and not accounting for clustering in the primary analysis. The aim of this review was to evaluate approaches for handling missing data and statistical analysis with respect to the primary outcome in CRTs.MethodsWe systematically searched for CRTs published between August 2013 and July 2014 using PubMed, Web of Science, and PsycINFO. For each trial, two independent reviewers assessed the extent of the missing data and method(s) used for handling missing data in the primary and sensitivity analyses. We evaluated the primary analysis and determined whether it was at the cluster or individual level.ResultsOf the 86 included CRTs, 80 (93 %) trials reported some missing outcome data. Of those reporting missing data, the median percent of individuals with a missing outcome was 19 % (range 0.5 to 90 %). The most common way to handle missing data in the primary analysis was complete case analysis (44, 55 %), whereas 18 (22 %) used mixed models, six (8 %) used single imputation, four (5 %) used unweighted generalized estimating equations, and two (2 %) used multiple imputation. Fourteen (16 %) trials reported a sensitivity analysis for missing data, but most assumed the same missing data mechanism as in the primary analysis. Overall, 67 (78 %) trials accounted for clustering in the primary analysis.ConclusionsHigh rates of missing outcome data are present in the majority of CRTs, yet handling missing data in practice remains suboptimal. Researchers and applied statisticians should carry out appropriate missing data methods, which are valid under plausible assumptions in order to increase statistical power in trials and reduce the possibility of bias. Sensitivity analysis should be performed, with weakened assumptions regarding the missing data mechanism to explore the robustness of results reported in the primary analysis.Electronic supplementary materialThe online version of this article (doi:10.1186/s13063-016-1201-z) contains supplementary material, which is available to authorized users.

Review of Recent Methodological Developments in Group-Randomized Trials: Part 2—Analysis

“…Two recent reviews 6,96 indicate that missing outcome data is common in GRTs, though investigators frequently analyze only available data without accounting for the missing data pattern. When the covariate-dependent missingness (CDM) assumption is plausible, both mixed effects and GEE models provide unbiased estimates of the intervention effect when the CDM covariates are included in an analysis of all available data.…”

“…101 A recent review showed that very few GRTs performed any sensitivity analyses for their missing data assumptions. 6 …”

“…1–5 For analysis, regression modeling approaches are generally preferred and most commonly used because of their ease of implementation. 6 Several textbooks now address these and other issues. 1–5 …”

Missing data handling in non‐inferiority and equivalence trials: A systematic review

“…These reviews show that even in high impact journals, there remain problems with reporting on the methodological approach, interpretation of the result of NI, and on the choice of the NI margin . Meanwhile, reviews of missing data in randomized controlled trials (RCTs) report that missing data are common, and that simple methods for handling missing data, although lacking in justification, remain popular . Furthermore, contrary to National Research Council recommendations, typically very few sensitivity analyses are carried out .…”

Using generalized estimating equations and extensions in randomized trials with missing longitudinal patient reported outcome data

“…Thus, researchers should be aware of analytical options for handling missing data robustly. While there is some evidence that researchers are beginning to use more principled methods, including multiple imputation and maximum likelihood based mixed models, when analyzing PROs and other outcomes in randomized controlled trials (RCTs), many use analyses that can result in bias, including single imputation and complete case analyses . One rarely used class of methods, which includes extensions for missing data, is generalized estimating equations (GEEs) …”