The fragility of statistically significant results from clinical nutrition randomized controlled trials

Pędziwiatr, Michał; Mizera, Magdalena; Wysocki, Michał; Małczak, Piotr; Stefura, Tomasz; Gajewska, Natalia; Torbicz, Grzegorz; Droś, Jakub; Kacprzyk, Artur; Major, Piotr; Kłęk, Stanisław; Bała, Małgorzata M

doi:10.1016/j.clnu.2019.05.024

Cited by 6 publications

(5 citation statements)

References 55 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Particularly in longer studies with variable follow-up time periods, analyses that account for time (such as a Kaplan-Meier curve, or a Cox proportional hazards model) are more appropriate than a simple binary outcome analysis. Fourth, our study shows a tendency of the inverse correlation between the FI and p-value, which is similar with previous FI studies [19,20]. This might be the RCT studies included small number patients.…”

Section: Resultssupporting

confidence: 91%

Frequent fragility of randomized controlled trials for HCC treatment

Hao

Jingtao

Zeng

2020

Preprint

View full text Add to dashboard Cite

Background: The fragility index (FI) of trial results can provide a measure of confidence in the positive effects reported in randomized controlled trials (RCTs). The aim of this study was to calculate the FI of RCTs supporting HCC treatments.Methods: A methodological systematic review of RCTs in HCC treatments was conducted. Two-arm studies with randomized and positive results for a time-to-event outcome were eligible for the FI calculation.Results: A total of 6 trails were included in this analysis. The median FI was 0.5 (IQR 0-10). FI was ≤ 7 in 4 (66.7%) of 6 trials; in those trials the fragility quotient was ≤ 1%.Conclusion: Many phase 3 RCTs supporting HCC treatments have a low FI, which challenges the confidence in concluding the superiority of these drugs over control treatments.

show abstract

Section: Resultssupporting

confidence: 91%

Frequent fragility of randomized controlled trials for HCC treatment

Hao

Jingtao

Zeng

2020

Preprint

View full text Add to dashboard Cite

show abstract

“…3 FI and FQ in the included studies hazards model) are more appropriate than a simple binary outcome analysis. Fourth, our study shows a tendency of the inverse correlation between the FI and p-value, which is similar with previous FI studies [18,19]. This might be the RCT studies included small number patients.…”

Section: Discussionsupporting

confidence: 91%

Frequent fragility of randomized controlled trials for HCC treatment

Zhang

Zeng

2021

BMC Cancer

View full text Add to dashboard Cite

Background The fragility index (FI) of trial results can provide a measure of confidence in the positive effects reported in randomized controlled trials (RCTs). The aim of this study was to calculate the FI of RCTs supporting HCC treatments. Methods A methodological systematic review of RCTs in HCC treatments was conducted. Two-arm studies with randomized and positive results for a time-to-event outcome were eligible for the FI calculation. Results A total of 6 trails were included in this analysis. The median FI was 0.5 (IQR 0–10). FI was ≤7 in 4 (66.7%) of 6 trials; in those trials the fragility quotient was ≤1%. Conclusion Many phase 3 RCTs supporting HCC treatments have a low FI, which challenges the confidence in concluding the superiority of these drugs over control treatments.

show abstract

“…The low robustness calculated herein indicates that even the best level of primary MD evidence proving causality, namely the RCTs, can fail to reach the standards one would expect. Recently, a study 76 assessing the FI of clinical nutrition trials revealed a low FI. According to Zeilstra, 77 many nutritional RCTs yield ambiguous results, which is why the RCT design is often considered 'ill-suited' for nutritional research.…”

Section: Discussionmentioning

confidence: 99%

“…Given the demand for evidence-based nutrition recommendations, [88][89][90][91] the results suggest that the formulation of recommendations promoting the MD based on RCTs should be performed with caution. 76 Thorough examination of the American College of Gastroenterology guidelines revealed that most RCTs used to guide recommendations regarding Crohn's disease relied on a small number of superior events for 'securing' statistical significance. 92 Often, the FI coincided with the drop-outs reported in some trials.…”

Section: Discussionmentioning

confidence: 99%

How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials

Grammatikopoulou

Nigdelis

Theodoridis

et al. 2021

BMJNPH

View full text Add to dashboard Cite

IntroductionThe Mediterranean diet (MD) is a traditional regional dietary pattern and a healthy diet recommended for the primary and secondary prevention of various diseases and health conditions. Results from the higher level of primary evidence, namely randomised controlled trials (RCTs), are often used to produce dietary recommendations; however, the robustness of RCTs with MD interventions is unknown.MethodsA systematic search was conducted and all MD RCTs with dichotomous primary outcomes were extracted from PubMed. The fragility (FI) and the reverse fragility index (RFI) were calculated for the trials with significant and non-significant comparisons, respectively.ResultsOut of 27 RCTs of parallel design, the majority failed to present a significant primary outcome, exhibiting an FI equal to 0. The median FI of the significant comparisons was 5, ranging between 1 and 39. More than half of the comparisons had an FI <5, indicating that the addition of 1–4 events to the treatment arm eliminated the statistical significance. For the comparisons with an FI=0, the RFI ranged between 1 and 29 (Median RFI: 7). When the included RCTs were stratified according to masking, the use of a composite primary endpoint, sample size, outcome category, or dietary adherence assessment method, no differences were exhibited in the FI and RFI between groups, except for the RFI among different compliance assessment methods.ConclusionsIn essence, the present study shows that even in the top tiers of evidence hierarchy, research on the MD may lack robustness, setting concerns for the formulation of nutrition recommendations.

show abstract

The fragility of statistically significant results from clinical nutrition randomized controlled trials

Cited by 6 publications

References 55 publications

Frequent fragility of randomized controlled trials for HCC treatment

Frequent fragility of randomized controlled trials for HCC treatment

Frequent fragility of randomized controlled trials for HCC treatment

How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials

Contact Info

Product

Resources

About