The Fragility of Statistically Significant Results in Pediatric Orthopaedic Randomized Controlled Trials as Quantified by the Fragility Index: A Systematic Review

Khormaee, Sariah; Choe, Julie C; Ruzbarsky, Joseph J.; Agarwal, Kavita; Blanco, John S.; Doyle, Shevaun M.; Dodwell, Emily

doi:10.1097/bpo.0000000000001201

Cited by 58 publications

(58 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Subsequently, any synthesis of related RCTs, although it may present low heterogeneity, carries an inherited risk of extrapolated findings. To nutrition's defence however, lower median FI compared with that of MD interventions has been reported in perioperative, 81 anesthesiology, 82 plastic surgery, 83 and critical-care medicine 84 RCTs, as well as among paediatric orthopedic 85 and appendicitis 86 trials. Nevertheless, the synthesis of these trials for recommendations formulation consists of a common practice in the fields mentioned above, as in the science of nutrition.…”

Section: Discussionmentioning

confidence: 99%

How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials

Grammatikopoulou

Nigdelis

Theodoridis

et al. 2021

BMJNPH

View full text Add to dashboard Cite

IntroductionThe Mediterranean diet (MD) is a traditional regional dietary pattern and a healthy diet recommended for the primary and secondary prevention of various diseases and health conditions. Results from the higher level of primary evidence, namely randomised controlled trials (RCTs), are often used to produce dietary recommendations; however, the robustness of RCTs with MD interventions is unknown.MethodsA systematic search was conducted and all MD RCTs with dichotomous primary outcomes were extracted from PubMed. The fragility (FI) and the reverse fragility index (RFI) were calculated for the trials with significant and non-significant comparisons, respectively.ResultsOut of 27 RCTs of parallel design, the majority failed to present a significant primary outcome, exhibiting an FI equal to 0. The median FI of the significant comparisons was 5, ranging between 1 and 39. More than half of the comparisons had an FI <5, indicating that the addition of 1–4 events to the treatment arm eliminated the statistical significance. For the comparisons with an FI=0, the RFI ranged between 1 and 29 (Median RFI: 7). When the included RCTs were stratified according to masking, the use of a composite primary endpoint, sample size, outcome category, or dietary adherence assessment method, no differences were exhibited in the FI and RFI between groups, except for the RFI among different compliance assessment methods.ConclusionsIn essence, the present study shows that even in the top tiers of evidence hierarchy, research on the MD may lack robustness, setting concerns for the formulation of nutrition recommendations.

show abstract

Section: Discussionmentioning

confidence: 99%

How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials

Grammatikopoulou

Nigdelis

Theodoridis

et al. 2021

BMJNPH

View full text Add to dashboard Cite

show abstract

“…The FI for virological cure by day 4 is 1, 2 for day 5, and 4 for day 6 [ Tables 3-5]. [11]. In the field of ophthalmology, the median index is 2 across 156 published trials [10].…”

Section: Resultsmentioning

confidence: 99%

“…Evaniew et al reviewed 40 orthopedic spine surgery trials and reported a median FI of 2 [8]. Pediatric orthopedic trials have a median FI of 3, according to a review by Khormaee et al of 17 trials [11]. In the field of ophthalmology, the median index is 2 across 156 published trials [10].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

How robust are the results of one of the first positive trials exploring hydroxychloroquine for treatment of COVID-19?

Chow

Elsayed

Lock

2020

Preprint

View full text Add to dashboard Cite

An outbreak of a novel human coronavirus infection emerged in Wuhan, China in December 2019. Two months later, the World Health Organization (WHO) announced SARS-CoV-2 as the name for the new virus and COVID-19 for the associated illness. On March 12, 2020, the WHO officially declared COVID-19 as a pandemic. The scientific community has raced to find effective therapeutic agents against the virus. Gautret et al 2020 is among one of the first purportedly positive trials of hydroxychloroquine for the treatment of COVID-19. However, it is imperative that a thorough analysis and understanding of trial data be undertaken prior to making claims about safety and efficacy. Our group assessed the statistical robustness of the trial using the Fragility Index (FI). The FI provides a numerical quantification of a clinical trial's conclusions. The index is based on iterative statistical calculations to determine the minimum number of events within a trial that would theoretically need to change from positive to negative in order for the trial's endpoint to convert from significant to non-significant; the higher the index, the more statistically robust the study results. For the Gautret et al trial, one endpoint had an FI of 1, two had indices of 2, and another had an index of 4. The primary endpoint of viral clearance on day 6 had an FI of 4. This indicates that if 4 events were to change from positive to negative, the conclusion of the trial would become mathematically non-significant. This index is comparable to many other published trials of established agents; the median FI across the reported literature appears to be 2. In conclusion, the trial results reported by Gautret et al are statistically robust, assuming that data quality is not compromised; however, the study was an open-label trial with non-homogenous groups, with analysis conducted per-protocol.Additionally, SARS-CoV-2 Reverse Transcriptase-PCR (RT-PCR) testing was not conducted in a systematic way amongst the two groups. Further analyses of this trial and future trials of All rights reserved. No reuse allowed without permission.(which was not certified by peer review) is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity.

show abstract

“…Evaniew et al reviewed 40 orthopedic spine surgery trials and reported a median FI of 2(9). Pediatric orthopedic trials have a median FI of 3, as reported in a review of 17 trials by Khormaee et al(12). In the field of ophthalmology, the median index was 2 across 156 published trials(11).…”

mentioning

confidence: 98%

Hydroxychloroquine for the treatment of COVID-19: the importance of scrutiny of positive trials

Chow¹,

Simone²,

Lock³

2020

Ann Palliat Med

View full text Add to dashboard Cite

Coronaviruses primarily target the human respiratory system. Over the past two decades, these viruses were responsible for three epidemics. Prior to 2019, the two coronavirus public health threats were caused by the Severe Acute Respiratory Syndrome (SARS-CoV) and Middle East Respiratory Syndrome (MERS-CoV) coronaviruses (1). In December 2019, an outbreak of a novel coronavirus was reported in Wuhan, China (2) with subsequent rapid spread around the world. On March 12, 2020, the World Health Organization (WHO) officially declared COVID-19 a pandemic. At the time of this writing in July 2020, there have been over 617,000 deaths worldwide (3), and the virus has required a significant role from palliative medicine providers to provide supportive care to patients and families impacted by COVID-19.Although the scientific community is racing to find possible active agents for COVID-19, there currently is no effective treatment (4). With no effective treatment, supportive care is the only option for patients, caregivers and family; as COVID-19 is known to produce significant symptom burden across multiple organ systems, multidisciplinary care will need to be expended while a treatment is being developed. Dexamethasone may be effective, but only in a subset of hospitalized patients (5). Remdesivir, which was developed to treat other viral diseases including Ebola, may also be effective. One of the first purportedly positive trials was published by Gautret et al. (6), a small, single-arm study reporting hydroxychloroquine to be a potentially-promising treatment to significantly reduce the viral carriage. Earlier versions of the results were released previously due to the urgent need to provide data

show abstract

The Fragility of Statistically Significant Results in Pediatric Orthopaedic Randomized Controlled Trials as Quantified by the Fragility Index: A Systematic Review

Abstract: Level I.

Cited by 58 publications

References 41 publications

How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials

How fragile are Mediterranean diet interventions? A research-on-research study of randomised controlled trials

How robust are the results of one of the first positive trials exploring hydroxychloroquine for treatment of COVID-19?

Hydroxychloroquine for the treatment of COVID-19: the importance of scrutiny of positive trials

Contact Info

Product

Resources

About