Transparency in Cognitive Training Meta-analyses: A Meta-review

Abstract

Meta-analyses often present flexibility regarding their inclusion criteria, outcomes of interest, statistical analyses, and assessments of the primary studies. For this reason, it is necessary to transparently report all the information that could impact the results. In this meta-review, we aimed to assess the transparency of meta-analyses that examined the benefits of cognitive training, given the ongoing controversy that exists in this field. Ninety-seven meta-analytic reviews were included, which examined a wide range of populations with different clinical conditions and ages. Regarding the reporting, information about the search of the studies, screening procedure, or data collection was detailed by most reviews. However, authors usually failed to report other aspects such as the specific meta-analytic parameters, the formula used to compute the effect sizes, or the data from primary studies that were used to compute the effect sizes. Although some of these practices have improved over the years, others remained the same. Moreover, examining the eligibility criteria of the reviews revealed a great heterogeneity in aspects such as the training duration, age cut-offs, or study designs that were considered. Preregistered meta-analyses often specified poorly how they would deal with the multiplicity of data or assess publication bias in their protocols, and some contained non-disclosed deviations in their eligibility criteria or outcomes of interests. The findings shown here, although they do not question the benefits of cognitive training, illustrate important aspects that future reviews must consider.

Preregistration of psychology meta-analyses: A cross-sectional study of prevalence and practice

“…To our knowledge, these findings provide the first field-wide examination of the prevalence and practice of preregistration in psychology meta-analyses. Other studies have provided a range of estimates (2%-36%) for the prevalence of meta-analysis preregistration in various sub-fields of psychology (López-Nicolás et al, 2022;Polanin et al, 2020;Sandoval-Lentisco et al, 2024), but direct comparison is difficult because of differences in time-frame. Comparing the prevalence measure in the current study (27%) to a recent field-wide estimate (8%) for the prevalence of preregistration in empirical psychology (Hardwicke et al, in preparation), suggests that preregistration is more common for meta-analyses relative to empirical studies in general in psychology.…”

“…López-Nicolás et al (2022) found that 19% of a random sample of 100 meta-analyses on clinical psychological interventions published between 2000 and 2023 were preregistered. Sandoval-Lentisco et al, (2024) found that 36% of 97 meta-analyses on cognitive training published between 1997 and 2023 (of which 40 were published from 2020) were preregistered. Currently, there is no available estimate for the prevalence of preregistered meta-analyses across the entire field of psychology.…”