The Unknown Prevalence of Postrandomization Bias in 15 Physical Therapy Journals: A Methods Review

Abstract

Objectives To determine the prevalence of prospective clinical trial registration and postrandomization bias in published musculoskeletal physical therapy randomized clinical trials (RCTs). Design A methods review. Literature Search Articles indexed in MEDLINE and published between January 2016 and July 2020 were included. Study Selection Criteria Two independent blinded reviewers identified the RCTs using Covidence. We included RCTs related to musculoskeletal interventions that were published in International Society of Physiotherapy Journal Editors member journals. Data Synthesis Data were extracted independently for the variables of interest from the identified RCTs by 2 blinded reviewers. The data were presented descriptively or in frequency tables. Results One hundred thirty-eight RCTs were identified. One third of RCTs were consistent with their prospectively registered intent (49/138); consistency with prospectively registered intent could not be determined for two thirds (89/138) of the RCTs. Four RCTs (8%)reported inconsistent results with the primary aims and 7 (14%) with the outcomes from the prospective clinical trial registry, despite high methodological quality (Physiotherapy Evidence Database [PEDro] scale score). Differences between prospectively registered and non-prospectively registered RCTs for PEDro scale scores had a medium effect size (r = 0.30). Two of 15 journals followed their clinical trial registration policy 100% of the time; in 1 journal, the published RCTs were consistent with the clinical trial registration. Conclusion Postrandomization bias in musculoskeletal physical therapy RCTs could not be ruled out, due to the lack of prospective clinical trial registration and detailed data analysis plans. J Orthop Sports Phys Ther 2021;51(11):542–550. Epub 21 Sep 2021. doi:10.2519/jospt.2021.10491

“Trustworthiness,” confidence in estimated effects, and confidently translating research into clinical practice

“…A 2021 methodological SR of RCTs involving musculoskeletal physical therapy interventions published in ISPJE member journals demonstrated that it could not be determined if researchers followed their original research question, followed their methods, or reported their study consistent with the established research record 64.5% (89/138) of the time [7]. This was secondary to the RCTs being retrospectively registered, unregistered, or having unclear registrations.…”

“…This was secondary to the RCTs being retrospectively registered, unregistered, or having unclear registrations. Furthermore, of the ISPJE member journals that required prospective clinical trial registration, it was established that 8.2% (4/49) of the RCTs changed their primary research question, and 16.3% (8/49) changed their primary outcome measures [7]. In addition, four of these questionable publications were rated as having Physiotherapy Evidence Database (PEDro) scores of 7-8/10 [7].…”

“…This is important when translating research into practice recommendations using the Grading of Recommendations, Assessment, Development and Evaluations (GRADE) approach, as confidence in the estimated effect is established through moderate to high-quality evidence [9]. If fidelity with prospective clinical trial registration cannot be determined, RCTs have an unclear risk of bias, as any tool used to establish RCT quality cannot help since we are unsure if they were conducted as planned [7]. This is a significant threat to the trustworthiness of our highest levels of preprocessed research evidence used by clinicians in clinical practice.…”

Trials and tribulations of transparency related to inconsistencies between plan and conduct in peer‐reviewed physiotherapy publications: A methodology review

“…The overall quality and transparency of clinical research has also been called into question 7,8,23–28 . This is important because clinicians rely not only on the results from clinical trials but also on details of the trial related to generalizability in their own settings.…”

“…Assessing adherence to checklists and guidelines for the conduct and reporting of research is a significant step toward monitoring the profession's strides toward high‐quality, transparent and reproducible research. A review of randomized controlled trials (RCTs) focusing only on musculoskeletal physiotherapy interventions reported that a mere third of selected RCTs were consistent with their pretrial registration 28 . Whether these findings can be extrapolated to other types of interventions or related to other areas of research is unknown and should be investigated.…”