The combination of reporting bias and underpowered study designs has substantially exaggerated the motor learning benefits of self-controlled practice and enhanced expectancies: a meta-analysis

Abstract

Enhanced expectancies and autonomy-support through self-controlled practice conditions form the motivation pillar of OPTIMAL theory (Wulf & Lewthwaite, 2016). The influence of these practice variables on motor learning was recently evaluated in two separate meta-analyses. Both meta-analyses found that the published literature suggested a moderate and significant benefit on motor learning; however, evidence for reporting bias was found in both literatures. Although multiple bias-corrected estimates were reported in the self-controlled meta-analysis, there was no principled way to prefer one over the other. In the enhanced expectancies meta-analysis, the trim-and-filltechnique failed to correct the estimated effects. Here, we addressed these limitations by reanalyzing the data from both meta-analyses using robust Bayesian meta-analysis methods. Our reanalysis revealed that reporting bias substantially exaggerated the benefits of these practice variables in the original meta-analyses. The true effects are instead small, uncertain, and potentially null. We also found the estimated average statistical power among all studies from the original meta-analyses was 6% (95% confidence interval [5%, 13%]). These results provide compelling and converging evidence that strongly suggests the available literature is insufficient to support the motivation pillar of OPTIMAL theory. Our results highlight the need for adequately powered experimental designs if motor learning scientists want to make evidence-based recommendations.

Untrustworthy Evidence in Dishonesty Research

“…The z-curve estimates for research on dishonesty are comparable to recently published estimates on motor learning benefits (McKay et al, 2023), effects of valenced odors on face perception and evaluation (Syrjänen et al, 2021), terror management theory (Chen et al, 2023), or system justification (Sotola & Credé, 2022). However, the general estimate for social psychology (Bartoš & Schimmack, 2022) or construal level theory (Maier et al, 2022) seems to be slightly better.…”