The Fragility Index in peri‐operative randomised trials that reported significant mortality effects in adults

Abstract

More than one million peri-operative patients die each year. Thus, small improvements in peri-operative care may save thousands of lives. However, clinicians need confidence in the robustness of trial findings. The Fragility Index may complement frequentist analysis and provide quantitative assessment of robustness. We searched MEDLINE for peri-operative critical care randomised controlled trials that reported a statistically significant difference in mortality. We identified 46 trials with 37,347 participants. The median (IQR [range]) Fragility Index was 2 (1-3 [0-49]). Eleven trials had a Fragility Index of zero (changing from the Chi-square test to Fisher's exact test removed significance) and seven trials had a Fragility Index of 1. Only 23/46 trials had a Fragility Index greater than the number of patients lost to follow-up. There was a strong positive correlation between the Fragility Index and: the number of participants, R 2 = 0.97, p < 0.0001; the number of centres that recruited participants, R 2 = 0.96, p < 0.0001; the number of nations that recruited participants, R 2 = 0.93, p < 0.0001; and the number of deaths, R 2 = 0.97, p < 0.0001. As measured by the Fragility Index, the effect of perioperative interventions on mortality in individual randomised controlled trials are not robust.

A randomised controlled trial of shoulder block vs. interscalene brachial plexus block for ventilatory function after shoulder arthroscopy

“…Our study was small although consistent with our sample size calculation and we believe it would have been inappropriate to recruit more participants . The fragility index for our study is 4, above the 75th percentile of a recent systematic review of 43 trials . We acknowledge that we did not design the trial to test the secondary outcomes, which should be considered hypothesis generating.…”

Are most randomised trials in anaesthesia and critical care wrong? An analysis using Bayes’ theorem

“…The index can be used for positive randomised trials with dichotomous outcomes when there are two groups allocated on a 1:1 ratio. In the peri-operative and critical care literature, the median Fragility Index where mortality was the primary outcome measure is two, with over 40% of trials having a Fragility Index of one or zero [12,13]. In more than half of the peri-operative trials, the Fragility Index is less than the number of patients lost to follow-up [12].…”

“…In the peri-operative and critical care literature, the median Fragility Index where mortality was the primary outcome measure is two, with over 40% of trials having a Fragility Index of one or zero [12,13]. In more than half of the peri-operative trials, the Fragility Index is less than the number of patients lost to follow-up [12]. Even for large, well-conducted studies such as B-Aware and POISE, the Fragility Index is only one and two respectively [14,15].…”

Negative outcomes in critical care trials: applying the wrong statistics – or asking the wrong questions?

“…There are two things we can do: one easy and one impossible. The easy one is, as Sidebotham discusses, to require all trials to report their ‘Fragility Index’ . Limitations of p values and confidence intervals have been widely discussed, yet misinterpretation continues.…”

“…There are two things we can do: one easy and one impossible. The easy one is, as Sidebotham discusses, to require all trials to report their 'Fragility Index' [13].…”