The Statistical Fragility of Single-Bundle vs Double-Bundle Autografts for ACL Reconstruction: A Systematic Review of Comparative Studies

Abstract

Background: The statistical significance of a given study outcome can be liable to small changes in findings. P values are common, but imperfect statistical methods to convey significance, and inclusion of the fragility index (FI) and fragility quotient (FQ) may provide a clearer perception of statistical strength. Purpose/Hypothesis: The purpose was to examine the statistical stability of studies comparing primary single-bundle to double-bundle anterior cruciate ligament reconstruction (ACLR) utilizing autograft and independent tunnel drilling. It was hypothesized that the study findings would be vulnerable to a small number of outcome event reversals, often less than the number of patients lost to follow-up. Study Design: Systematic review; Level of evidence, 2. Methods: Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, the authors searched PubMed for comparative studies and randomized controlled trials (RCTs) published in select journals, based on impact factor, between 2005 and 2020. Risk-of-bias assessment and methodology scoring were conducted for the included studies. A total of 48 dichotomous outcome measures were examined for possible event reversals. The FI for each outcome was determined by the number of event reversals necessary to alter significance. The FQ was calculated by dividing the FI by the respective sample size. Results: Of the 1794 studies screened, 15 comparative studies were included for analysis; 13 studies were RCTs. Overall, the mean FI and FQ were 3.14 (IQR, 2-4) and 0.050 (IQR, 0.032-0.062), respectively. For 72.9% of outcomes, the FI was less than the number of patients lost to follow-up. Conclusion: Studies comparing single-bundle versus double-bundle ACLR may not be as statistically stable as previously thought. Comparative studies and RCTs are at substantial risk for statistical fragility, with few event reversals required to alter significance. The reversal of fewer than 4 outcome events in a treatment group can alter the statistical significance of a given result; this is commonly less than the number of patients lost to follow-up. Future comparative study analyses might consider including FI and FQ with P values in their statistical analysis.

The Statistical Fragility of Operative vs Nonoperative Management for Achilles Tendon Rupture: A Systematic Review of Comparative Studies

“…For the outcomes included in this analysis, the median FI was 2.5 (IQR 2-4), the mean FI was 2.9 (±1.58), the median FQ was 0.032 (IQR 0.012-0.069), and the mean FQ was 0.049 (±0.062). These values are consistent with the current orthopaedic literature reporting an average median FI of 3.81 10 – 14 , 17 , 18 , 21 , 23 , 29 – 31 , 36 , 38 , 42 , 47 – 53 , 55 …”

“…These values are consistent with the current orthopaedic literature reporting an average median FI of 3.81 10–14,17,18,21,23,29 –31,36,38,42,47 –53,55 –57 and FQ of 0.048. 10–12,18,21,…”

“…For the outcomes included in this analysis, the median FI was 2.5 (IQR 2-4), the mean FI was 2.9 (±1.58), the median FQ was 0.032 (IQR 0.012-0.069), and the mean FQ was 0.049 (±0.062). These values are consistent with the current orthopaedic literature reporting an average median FI of 3.81 [10][11][12][13][14]17,18,21,23,[29][30][31]36,38,42,[47][48][49][50][51][52][53][55][56][57] and FQ of 0.048. [10][11][12]18,21,23,31,[47][48][49][50][51]53 The hypothesis of this study was confirmed, with more than half (78%) of outcomes analyzed having a loss to follow-up greater than the fragility index for that outcome.…”

“…These values are consistent with the current orthopaedic literature reporting an average median FI of 3.81 [10][11][12][13][14]17,18,21,23,[29][30][31]36,38,42,[47][48][49][50][51][52][53][55][56][57] and FQ of 0.048. [10][11][12]18,21,23,31,[47][48][49][50][51]53 The hypothesis of this study was confirmed, with more than half (78%) of outcomes analyzed having a loss to follow-up greater than the fragility index for that outcome. Furthermore, outcomes from studies with a greater risk of bias and rerupture as an individual outcome were more fragile than the other outcomes observed in this analysis.…”

Predictors of Increased Fragility Index Scores in Surgical Randomized Controlled Trials: An Umbrella Review

“…Twelve reviews reported FI values stratified by primary vs secondary outcome type. Two studies reported no statistically significant difference between primary and secondary outcome FI values [18, 19], while one study did report a significant difference [14]. Nine reviews investigated the relationship between FI values and journal impact factor.…”