The Replication Dilemma: Potential Challenges in Measuring Replication Value—A Commentary on Isager, Van’t Veer, & Lakens (2024)

Abstract

The authors (Isager, Van‘t Veer, & Lakens, 2024) start with the main assumption that researchers' efforts toward replications are constrained by resources, and they propose a simple, practically scalable framework of research replication value that guides the researchers and the scientific community at large intending to achieve bigger bang for the buck. Specifically, the authors propose a framework that combines citation impact and sample size of the original articles as a metric for assessing replication value. This implies that original studies with higher scores on this metric can be prioritized for replication efforts. We thoroughly agree with the authors' assumption, and indeed support the view of working towards an optimal framework that helps the community achieve maximum research impact from the replication efforts. In this commentary, we propose to discuss three important limitations that have to be considered before using such metrics.