The Statistical Fragility of Hamstring Versus Patellar Tendon Autografts for Anterior Cruciate Ligament Reconstruction: A Systematic Review of Comparative Studies

Abstract

Background: Evidence-based medicine utilizes data to inform clinical decision making, despite the ability of a small number of outcome reversals to change statistical significance. P values are common measurements of statistical significance that possess inherent flaws. The inclusion of the fragility index (FI) and fragility quotient (FQ) may provide a clearer conveyance of statistical strength. Purpose/Hypothesis: The purpose was to examine the statistical stability of studies comparing hamstring tendon and bone–patellar tendon–bone autografts in primary single-bundle anterior cruciate ligament reconstruction with independent tunnel drilling. We hypothesized that the findings of these studies are vulnerable to a small number of outcome event reversals, often fewer than the number of patients lost to follow-up. Study Design: Systematic review. Methods: Comparative studies and randomized controlled trials (RCTs) published in 10 leading orthopaedic journals between 2000 and 2020 were analyzed. Statistical significance was defined as a P value ≤.05. FI for each outcome was determined by the number of event reversals necessary to alter significance. FQ was calculated by dividing the FI by the respective sample size. Results: Of the 1803 studies screened, 643 met initial search criteria, with 18 comparative studies ultimately included for analysis, 8 of which were RCTs. A total of 114 outcomes were examined. Overall, the mean (interquartile range) FI and FQ were 3.77 (2-4) and 0.040 (0.016-0.055), respectively. The FI was less than the number of patients lost to follow-up for 76.3% of outcomes. Conclusion: Studies examining graft choice for anterior cruciate ligament reconstruction may not be as statistically stable as previously thought. Comparative studies and RCTs are at substantial risk for statistical fragility, with few event reversals required to alter significance. The reversal of <4 outcome events in a treatment group can alter the statistical significance of a given result; this is commonly fewer than the number of patients lost to follow-up. Future comparative study analyses might consider including FI and FQ with P values in their statistical analysis.

Statistical Fragility of Single-Row Versus Double-Row Anchoring for Rotator Cuff Repair: A Systematic Review of Comparative Studies

“…However, this study had a very large patient population compared with the average orthopaedic study, with 5540 patients. By dividing the FI by the 5540 patients included in this population database study, Ehlers et al 9 found an FQ of 0.021, which is more consistent with the average FQ reported in orthopaedic literature (0.036) and in this study (0.043). This example highlights how FI can be misinterpreted when used on its own.…”

“…For this review, the median FI was found to be 2, and the median FQ was 0.035, which is consistent with previous orthopaedic literature reporting an average median FI of 3.67 ¶ and FQ of 0.036. 9 , 19 , 27 , 35 – 37 , 45 This study’s findings demonstrated that statistical significance can be changed with <3 outcome reversals, or reversal of 3.4% of outcome events. Furthermore, 13 of the 16 significant outcomes (81%) reported needed just a single outcome reversal to become statistically nonsignificant.…”

“… 18 Currently, the most frequently used techniques are the single-row (SR) and double-row (DR) repairs. 9 However, the designations “single row” and “double row” are inherently broad and encompass techniques such as the modified Mason-Allen (MMA) stitch, triple-loaded anchor repair (TLAR), and classic SR techniques for SR repairs, and the transosseous equivalent (TOE) and classic DR techniques for DR repairs. 30 …”

“… 9 Recently, the literature has suggested that DR repair may offer a more biomechanically sound construct, but it is more costly and time demanding than is SR repair. 2 , 9 The surgeon’s choice between SR and DR repair is guided primarily by evidence-based medicine, with an emphasis placed on these comparative studies and randomized controlled trials (RCTs). However, the statistical stability of these studies is subject to question and may be more fragile than previously thought.…”

“…18 Currently, the most frequently used techniques are the single-row (SR) and double-row (DR) repairs. 9 However, the designations "single row" and "double row" are inherently broad and encompass techniques such as the modified Mason-Allen (MMA) stitch, triple-loaded anchor repair (TLAR), and classic SR techniques for SR repairs, and the transosseous equivalent (TOE) and classic DR techniques for DR repairs. 30 Despite the wide range of techniques described in the literature, no consensus currently exists as to what should be the standard repair technique, and comparative studies between SR and DR constructs are ongoing.…”