Proposal of a Selection Protocol for Replication of Studies in Sports and Exercise Science

Abstract

Introduction To improve the rigor of science, experimental evidence for scientific claims ideally needs to be replicated repeatedly with comparable analyses and new data to increase the collective confidence in the veracity of those claims. Large replication projects in psychology and cancer biology have evaluated the replicability of their fields but no collaborative effort has been undertaken in sports and exercise science. We propose to undertake such an effort here. As this is the first large replication project in this field, there is no agreed-upon protocol for selecting studies to replicate. Criticism of previous selection protocols include claims they were non-randomised and non-representative. Any selection protocol in sports and exercise science must be representative to provide an accurate estimate of replicability of the field. Our aim is to produce a protocol for selecting studies to replicate for inclusion in a large replication project in sports and exercise science. Methods The proposed selection protocol uses multiple inclusion and exclusion criteria for replication study selection, including: the year of publication and citation rankings, research disciplines, study types, the research question and key dependent variable, study methods and feasibility. Studies selected for replication will be stratified into pools based on instrumentation and expertise required, and will then be allocated to volunteer laboratories for replication. Replication outcomes will be assessed using a multiple inferential strategy and descriptive information will be reported regarding the final number of included and excluded studies, and original author responses to requests for raw data.

Replication concerns in sports and exercise science: a narrative review of selected methodological issues in the field

“…Despite the core importance of replicability, very few replication studies have been attempted in sports and exercise science [ 33 ]. In this regard, it is worth mentioning a current collaborative replication project in the field attempting to conduct close replications of original study findings [ 106 ].…”

Exploring Equivalence Testing with the Updated TOSTER R Package

“…During the development of this TOSTER update, I was helping advise a team of researchers on a massive replication project for sport and exercise science (Murphy et al 2022). How to determine whether a direct 12 replication was a successful replication of the original study was contentious topic of conversation among the team.…”

Replication of null results: Absence of evidence or evidence of absence?

“…It is crucial to note that it is not our intention to diminish the enormously important contributions of the RPCB, the EPRP, and the RPP, but rather to build on their work and provide recommendations for ongoing and future replication projects (e.g. Amaral et al, 2019 ; Murphy et al, 2023 ).…”