Accuracy and precision of fixed and random effects in meta‐analyses of randomized control trials for continuous outcomes

Abstract

Meta‐analyses of treatment effects in randomized control trials are often faced with the problem of missing information required to calculate effect sizes and their sampling variances. Particularly, correlations between pre‐ and posttest scores are frequently not available. As an ad‐hoc solution, researchers impute a constant value for the missing correlation. As an alternative, we propose adopting a multivariate meta‐regression approach that models independent group effect sizes and accounts for the dependency structure using robust variance estimation or three‐level modeling. A comprehensive simulation study mimicking realistic conditions of meta‐analyses in clinical and educational psychology suggested that imputing a fixed correlation 0.8 or adopting a multivariate meta‐regression with robust variance estimation work well for estimating the pooled effect but lead to slightly distorted between‐study heterogeneity estimates. In contrast, three‐level meta‐regressions resulted in largely unbiased fixed effects but more inconsistent prediction intervals. Based on these results recommendations for meta‐analytic practice and future meta‐analytic developments are provided.