The Fragility of Statistically Significant Findings From Randomized Controlled Trials in Hip and Knee Arthroplasty

Ekhtiari, Seper; Gazendam, Aaron; Nucci, Nicholas; Kruse, Colin; Bhandari, Mohit

doi:10.1016/j.arth.2020.12.015

Cited by 29 publications

(14 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This study contradicts that of Ekhtiari et al. that was recently published [ 11 ]. In it, the authors performed a literature search to identify RCTs performed for primary or revision surgery and ultimately included 34 RCTs from the past decade in TJA literature and found that the median FI was 1, meaning that reversing the outcome of just one subject would change any statistically significant outcome to not statistically significant.…”

Section: Discussioncontrasting

confidence: 99%

“…In evaluating the FI of both significant and nonsignificant outcomes, however, we found a much higher median FI of 6 overall, and 4 and 7 for significant and nonsignificant outcomes, respectively. Both these values are greater than what their study reported [ 11 ]. Our research evaluated different studies—we chose to evaluate solely primary TJA RCTs describing surgical interventions in the top 25 highest impact orthopedic journals, with manuscripts in English.…”

Section: Discussioncontrasting

confidence: 77%

“…A recent study by Ekhtiari et al. described the FI of statistically significant outcomes in 34 RCTs in total joint arthroplasty (TJA) [ 11 ]. However, their sample was small, and this article seeks to expand that search.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

Background: The Fragility Index (FI) and Reverse Fragility Index are powerful tools to supplement the P value in evaluation of randomized clinical trial (RCT) outcomes. These metrics are defined as the number of patients needed to change the significance level of an outcome. The purpose of this study was to calculate these metrics for published RCTs in total joint arthroplasty (TJA). Methods: We performed a systematic review of RCTs in TJA over the last decade. For each study, we calculated the FI (for statistically significant outcomes) or Reverse Fragility Index (for nonstatistically significant outcomes) for all dichotomous, categorical outcomes. We also used the Pearson correlation coefficient to evaluate publication-level variables. Results: We included 104 studies with 473 outcomes; 92 were significant, and 381 were nonstatistically significant. The median FI was 6 overall and 4 and 7 for significant and nonsignificant outcomes, respectively. There was a positive correlation between FI and sample size (R ¼ 0.14, P ¼ .002) and between FI and P values (R ¼ 0.197, P ¼ .000012). Conclusions: This study is the largest evaluation of FI in orthopedics literature to date. We found a median FI that was comparable to or higher than FIs calculated in other orthopedic subspecialties. Although the mean and median FIs were greater than the 2 recommended by the American Academy of Orthopaedic Surgeons Clinical Practice Guidelines to demonstrate strong evidence, a large percentage of studies have an FI < 2. This suggests that the TJA literature is on par or slightly better than other subspecialties, but improvements must be made. Level of Evidence: Level I; Systematic Review.

show abstract

Section: Discussioncontrasting

confidence: 99%

Section: Discussioncontrasting

confidence: 77%

See 1 more Smart Citation

Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review

et al. 2021

View full text Add to dashboard Cite

show abstract

“…A trial can be methodologically sound but poorly reported and vice-versa. Although the most widely adopted, the Cochrane risk of bias tool is one of many ways clinicians can critically appraise randomized controlled trials [33]. Finally, a potential limitation of the current review is the utility of the Cochrane risk of bias tool.…”

Section: Discussionmentioning

confidence: 99%

Assessment of Risk of Bias in Osteosarcoma and Ewing’s Sarcoma Randomized Controlled Trials: A Systematic Review

et al. 2021

Self Cite

View full text Add to dashboard Cite

Aim: The aim of this study was to systematically assess the risk of bias in osteosarcoma and Ewing’s sarcoma (ES) randomized controlled trials (RCT) and to examine the relationships between bias and conflict of interest/industry sponsorship. Methods: An OVID-MEDLINE search was performed (1976–2019). Using the Cochrane Collaboration guidelines, two reviewers independently assessed the prevalence of risk of bias in different RCT design domains. The relationship between conflicts of interest and industry funding with the frequency of bias was examined. Results: 73 RCTs met inclusion criteria. Prevalence of low-risk bias domains was 47.3%, unclear-risk domains 47.8%, and 4.9% of the domains had a high-risk of bias. Domains with the highest risk of bias were blinding of participants/personnel and outcome assessors, followed by randomization and allocation concealment. Overtime, frequency of unclear-risk of bias domains decreased (χ2 = 5.32, p = 0.02), whilst low and high-risk domains increased (χ2 = 8.13, p = 0.004). Studies with conflicts of interest and industry sponsorships were 4.2 and 3.1 times more likely to have design domains with a high-risk of bias (p < 0.05). Conclusion: This study demonstrates that sources of potential bias are prevalent in both osteosarcoma and ES RCTs. Studies with financial conflicts of interest and industry sponsors were significantly more likely to have domains with a high-risk of bias. Improvements in reporting and adherence to proper methodology will reduce the risk of bias and improve the validity of the results of RCTs in osteosarcoma and ES.

show abstract

“…Five of the 10 RCTs had some concern for risk of bias found in their study. The average MINORS score for comparative studies was 14 (range [13][14][15][16].…”

Section: Resultsmentioning

confidence: 99%

The Statistical Fragility of Operative vs Nonoperative Management for Achilles Tendon Rupture: A Systematic Review of Comparative Studies

et al. 2022

View full text Add to dashboard Cite

Background: The statistical significance of randomized controlled trials (RCTs) and comparative studies is often conveyed utilizing the P value. However, P values are an imperfect measure and may be vulnerable to a small number of outcome reversals to alter statistical significance. The interpretation of the statistical strength of these studies may be aided by the inclusion of a Fragility Index (FI) and Fragility Quotient (FQ). This study examines the statistical stability of studies comparing operative vs nonoperative management for Achilles tendon rupture. Methods: A systematic search was performed of 10 orthopaedic journals between 2000 and 2021 for comparative studies focusing on management of Achilles tendon rupture reporting dichotomous outcome measures. FI for each outcome was determined by the number of event reversals necessary to alter significance ( P < .05). FQ was calculated by dividing the FI by the respective sample size. Additional subgroup analyses were performed. Results: Of 8020 studies screened, 1062 met initial search criteria with 17 comparative studies ultimately included for analysis, 10 of which were RCTs. A total of 40 outcomes were examined. Overall, the median FI was 2.5 (interquartile range [IQR] 2-4), the mean FI was 2.90 (±1.58), the median FQ was 0.032 (IQR 0.012-0.069), and the mean FQ was 0.049 (±0.062). The FI was less than the number of patients lost to follow-up for 78% of outcomes. Conclusion: Studies examining the efficacy of operative vs nonoperative management of Achilles tendon rupture may not be as statistically stable as previously thought. The average number of outcome reversals needed to alter the significance of a given study was 2.90. Future analyses may benefit from the inclusion of a fragility index and a fragility quotient in their statistical analyses.

show abstract

The Fragility of Statistically Significant Findings From Randomized Controlled Trials in Hip and Knee Arthroplasty

Cited by 29 publications

References 60 publications

Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review

Fragility Index as a Measure of Randomized Clinical Trial Quality in Adult Reconstruction: A Systematic Review

Assessment of Risk of Bias in Osteosarcoma and Ewing’s Sarcoma Randomized Controlled Trials: A Systematic Review

The Statistical Fragility of Operative vs Nonoperative Management for Achilles Tendon Rupture: A Systematic Review of Comparative Studies

Contact Info

Product

Resources

About