The fragility of statistically significant findings from randomised controlled trials in the urological literature

Narayan, Vikram M.; Gandhi, Shreyas; Chrouser, Kristin; Evaniew, Nathan; Dahm, Philipp

doi:10.1111/bju.14210

Cited by 38 publications

(18 citation statements)

References 60 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, an additional metric like the FI should be reported together with the results. On the other hand, a strong negative correlation between the FI and sample sizes was detected in some studies [27][28][29][30][31] . In our study, we could not detect such a correlation because most trials included in our review had small sample sizes.…”

Section: Discussionmentioning

confidence: 96%

“…An approach named Fragility Index was introduced by FI was calculated for different medical literatures as urologic, orthopedic surgery, giant cell arthritis, anesthesiology and etc. [29][30][31][32] . There was no research found, which evaluated the robustness of the results for randomized controlled trials using diagnostic imaging techniques and the Fisher exact test together before.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Evaluation of statistical significance of randomized controlled trials using diagnostic imaging techniques with fragility index

Yıldırım¹,

Taşdelen²

2019

Cukurova Medical Journal

View full text Add to dashboard Cite

ÖzPurpose: The aim of our study was to evaluate the robustness and success of fragility index (FI) for randomized controlled trials (RCT) utilized diagnostic imaging techniques. Materials and Methods: We performed a systematic survey of RCTs using the terms of "randomized controlled trial", "exact test" and "diagnostic imaging" in PubMed. Two researchers independently reviewed the abstracts and identified the studies according to selection criteria. Results: The median FI was 4.0 [1.0-4.0] and the median sample size was 83.5[36.0-148.0]. Fifty percent of RCTs reported p value was between 0.001 and 0.05. After one more event added, 94.4 % of studies became nonsignificant with a p value between 0.05 and 0.10. There was no significant correlation between the FI's and the sample sizes and outcome events. (r=0.144, p=0.570; r=0.169, p=0.504) There was a statistically significant difference between reported p value groups in terms of FI. Conclusion: FI should be given along with the p value for binary outcomes. Researchers should be careful with the interpretation of FI because it only shows the level of evidence and the power of the statistical significance.Amaç: Bu çalışmanın amacı kırılganlık indeksinin (Kİ) başarısını, tanı amaçlı görüntüleme yöntemleri kullanılan randomize kontrollü çalışmalar (RKÇ) için değerlendirmektir. Gereç ve Yöntem: Pubmed veri tabanında "Randomize Kontrollü Çalışma", "Kesin Test" ve "Tanısal Görüntüleme" terimlerini içeren bir sistematik araştırma yapılmıştır. İki araştırmacı birbirinden bağımsız olarak özetleri ve çalışmaları seçim kriterlerine göre incelemiştir. Bulgular: Kİ medyan değeri 4.0 [1.0-4.0] ve örnek genişliğinin medyan değeri 83.5[36.0-148.0] olarak bulunmuştur. Dahil edilen çalışmaların %50'sinin p değeri 0.001 ile 0.05 arasında yer almaktadır. Bir kişi daha eklendiği durumda çalışmaların %94,4'ünün p değeri anlamsız hale gelmiştir. Kİ ile örnek genişliği ve ilgilenilen olayın gözlendiği kişi sayısı arasında anlamlı bir ilişki yoktur. (r=0.144, p=0.570; r=0.169, p=0.504) Rapor edilen p değeri grupları arasında Kİ değerleri bakımından istatistiksel olarak anlmalı bir farklılık vardır. ( Sonuç: İkili sonuç değişkenleri için Kİ p değeri ile birlikte verilmelidir. Araştırmacılar Kİ'yi yorumlarken dikkatli olmalıdır çünkü Kİ sadece çalışmanın kanıt değerini ve istatistiksel gücünü göstermektedir.

show abstract

Section: Discussionmentioning

confidence: 96%

Section: Discussionmentioning

confidence: 99%

Evaluation of statistical significance of randomized controlled trials using diagnostic imaging techniques with fragility index

Yıldırım¹,

Taşdelen²

2019

Cukurova Medical Journal

View full text Add to dashboard Cite

show abstract

“…Narayan et al reported an FI of 3 (IQR 1-4.5), where 67.5% of the trials had a number of patients lost to follow-up greater than the FI in an analysis of RCTs in the field of urology [11]. Likewise, a study examining RCTs in spine surgery reported an FI of 2, where 25% of the trials had a number of patients lost to follow up greater than the FI [14].…”

Section: Discussionmentioning

confidence: 99%

“…Studies in the fields of urology and spine surgery have, however, reported a positive correlation between FI and sample size, while other studies have demonstrated a weak positive correlation. A study in pediatrics published in 2017 reported no correlation between FI and sample size [11][12][13][14][15]. The relationships between FI and sample size, along with other parameters, are inconsistent among several studies and a concrete understanding is yet to be established.…”

Section: Discussionmentioning

confidence: 99%

“…The FI can thus help to assess the robustness of the trial results that form the foundation for these guideline recommendations. The use of the FI was first shown to be of value in a paper published in 2014, which reported a median FI of 8 for approximately 400 RTCs, and since then it has gained traction as a novel metric in several specialties, such as critical care medicine, pediatrics, urology, and spinal surgery [10][11][12][13][14]. In fact, an article in Chest journal examined the strength of the trial outcomes used to create the 2018 CHEST guideline and expert panel report on antithrombotic therapy for venous thromboembolism disease [15].…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Reexamining the data used in the 2012 guidelines of the American Association for the Study of Liver Diseases for the management of adult patients with ascites due to cirrhosis

Agrawal

Majeed

Attar

et al. 2019

aog

View full text Add to dashboard Cite

Background In 2012, the American Association for the Study of Liver Diseases published practice guidelines for the management of patients with ascites caused by cirrhosis, using data from randomized controlled trials (RCTs) and observational studies. We reexamined the strength of these RCTs by calculating the fragility index (FI), a novel metric proposed for evaluating the robustness of RCTs. Methods We screened all RCTs referenced in the guidelines for specific criteria. We calculated the FI and fragility quotient (FQ), and analyzed the correlation between FI and several variables. Results Twenty-one RCTs were included. The median (25 th , 75 th) FI and FQ were 1 (interquartile range [IQR] 0.5-6) and 0.070 (IQR 0.008-0.166), respectively. For studies that reported the number of patients lost to follow up (12 RCTs), the median of patients lost was 2 (IQR 0-6.5). There was no significant correlation between FI and sample size (r s =0.357), P-value (r s =-0.299), number lost to follow up (r s =0.355), Science Citation Index (r s =0.347), year of publication (r s =-0.085), blinding (r pb =-0.18) or number of centers (r pb =0.10). However, a significant correlation was seen between FI and number needed to treat (r s =-0.549; P=0.015). Conclusions RCTs in the field of cirrhosis-related ascites are fragile. Of the 21 trials analyzed, 13 had an FI of 3 or below and these trials influenced 13 of the 49 recommendations in the guidelines. We recommend the incorporation of FI and FQ in addition to P-value to better understand the meaning of the results in gastroenterological studies.

show abstract

The Fragility Index in Randomized Clinical Trials as a Means of Optimizing Patient Care

2019

View full text Add to dashboard Cite

andomizedclinicaltrials(RCTs)serveasthegoldstandardand represent the highest level of evidence for the determination of optimal and effective treatment strategies in evidencebased medicine for patients, particularly in terms of statistical reliability. Methods used in RCTs decrease selection bias and minimize confounding, which promote their ability to establish causation.Clinicians evaluating RCT results in an effort to provide optimal patient care frequently rely on rejection of the null hypothesis, with a P value threshold of less than .05 and appropriate 95% CIs used to determine statistical significance and confirm a positive outcome. These positive findings from RCTs frequently lead to substantial changes in clinical practice and patient care. Yet methodologists have noted that medical research studies should not be interpreted from P values alone, [1][2][3][4] noting that a "high rate of non-replication (lack of confirmation) of research discoveries is a consequence of the convenient, yet ill-founded strategy of claiming conclusive research findings solely on the basis of a single study assessed by formal statistical significance, typically for a p-value less than 0.05." 2(pe124) The Fragility Index (FI) is an important aid in the interpretation of clinical trial results. 5 The FI aids in determination of when statistical sig-nificance in a clinical trial may be lost as a result of a shift of a few additional events from the experimental group to the control group. 6 No matter what threshold P value is chosen, the FI score will still serve as an additional metric to demonstrate how easily statistical significance may be exceeded. Use of the FI can aid physicians in identifying trials that may be at risk of being overturned by future studies and avoiding overestimating the significance of RCT results. This narrative review discusses the definition and calculation of the FI and Fragility Quotient (FQ) scores, use in the interpretation of medical and surgical RCTs, a detailed review of FI scores in trauma RCTs, and discussion of the importance of this statistic in the development of national clinical guidelines. It provides a framework for surgeons to use the FI in the assessment of RCTs to ultimately provide optimal surgical patient care.

show abstract

The fragility of statistically significant findings from randomised controlled trials in the urological literature

Cited by 38 publications

References 60 publications

Evaluation of statistical significance of randomized controlled trials using diagnostic imaging techniques with fragility index

Evaluation of statistical significance of randomized controlled trials using diagnostic imaging techniques with fragility index

Reexamining the data used in the 2012 guidelines of the American Association for the Study of Liver Diseases for the management of adult patients with ascites due to cirrhosis

The Fragility Index in Randomized Clinical Trials as a Means of Optimizing Patient Care

Contact Info

Product

Resources

About