Preregistration of psychology meta-analyses: A cross-sectional study of prevalence and practice

Abstract

Considering their influential role in synthesizing the existing evidence on a particular topic, it is especially important that meta-analyses are conducted and reported to the highest standards, and the risk of bias is minimized. Preregistration can help detect and reduce bias arising from opportunistic use of ‘researcher degrees of freedom’. However, little is known about the prevalence and practice of the preregistration of meta-analyses in psychology. In this study, we first measured the prevalence of preregistration in all psychology meta-analyses published in 2021. Next, for 100 randomly selected preregistered meta-analyses, we evaluated the preregistration’s coverage of key meta-analytic decisions and the extent to which published meta-analyses deviated from their preregistered protocols. Of all 1403 eligible psychology meta-analyses published in 2021, 382 (27%) were preregistered. In our random sample, we found that key PRISMA-P decision items were often omitted from preregistered protocols — out of the 23 decisions items that were examined, the median number of items covered was 13 (interquartile range [IQR] = 11 to 14). We also found that all 100 preregistered meta-analyses contained at least one deviation from the preregistered protocol (median = 9 deviations, IQR = 6.75 to 11), and most deviations were undisclosed (median = 8 undisclosed deviations, IQR = 6 to 11). These findings show that, although preregistration is useful to detect the risk of bias, its infrequent use and poor implementation in psychology meta-analyses undermines its potential benefits of reducing bias and increasing transparency.