BACKGROUND: In biostatistics, evaluating fragility is crucial for
understanding their vulnerability to miscategorization. One proposed
measure of statistical fragility is the unit fragility index (UFI),
which measures the susceptibility of the p-value to flip significance
with minor changes in outcomes. Although the UFI provides valuable
information, it relies on p-values, which are arbitrary measures of
statistical significance. Alternative measures, such as the fragility
quotient (FQ) and the percent fragility index, have been proposed to
decrease the UFI’s reliance on sample size. However, these approaches
still rely on p-values and thus depend on an arbitrary cutoff of p
< 0.05. Instead of quantifying fragility by relying on
p-values, this study evaluated the effect of small changes on relative
risk. METHODS: Random 2x2 contingency tables associated with an initial
p-value of 0.001 to 0.05 were evaluated. Each table’s UFI and relative
risk index (RRI) were calculated. A derivative of the RRI, the percent
RRI, was also calculated along with the FQ. The UFI, FQ, RRI, pRRI,
initial p-value, and sample size were compared. RESULTS: A total of
15000 cases were tested. The correlation between the UFI and the p-value
was the strongest (r = -0.807), and the correlation between the pRRI was
the weakest (r = -0.395). The RRI had the strongest correlation with the
sample size (r = 0.826), and the UFI had the weakest correlation (r =
0.3904). The coefficient of variation for the average RRI was the
smallest at 28.3%, and for the FQ, it was the greatest at 57.0%. The
correlation between the UFI, FQ, and p-value is significantly greater
than the correlation between the RRI, pRRI, and p-value (for all
comparisons, p < 0.001). CONCLUSION: The RRI and pRRI are
significantly less correlated with the p-value than the UFI and FQ,
indicating relative independence of the RRI and pRRI from p-values.