Spin occurs in bariatric surgery randomized controlled trials with a statistically nonsignificant primary outcome: A systematic review

Rassy, Nathalie Al; Rives‐Lange, Claire; Carette, Claire; Barsamian, Charles; Moszkowicz, David; Thereaux, Jéremie; Poghosyan, Tigran; Czernichow, Sébastien

doi:10.1016/j.jclinepi.2021.05.004

Cited by 11 publications

(3 citation statements)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It was reported that the authors added spin in the manuscripts requested by peer reviewers in 9% of the publications, and peer reviewers failed to identify spin in the abstract conclusions in 76% of the publications [Lazarus et al, 2016]. Several previously published methodological studies on spin in the abstracts of RCTs with statistically nonsignificant primary outcomes also reported a high prevalence of spin in various fields of medicine [Boutron et al, 2010;Arunachalam et al, 2017;Eleftheriadi et al, 2020;Wu et al, 2020;Guo et al, 2021;Rassy et al, 2021]. Among those previous studies, several studies assessed spin in dentistry.…”

Section: Interpretation Of the Main Findingsmentioning

confidence: 99%

“…Several methodological studies have assessed spin in the titles and abstracts of RCTs in medicine, including dentistry [Boutron et al, 2010;Lockyer et al, 2013;Arunachalam et al, 2017;Khan et al, 2019;Eleftheriadi et al, 2020;Wu et al, 2020;Guo et al, 2021;Ito et al, 2021;Rassy et al, 2021]. The prevalence of spin in those reviews was high, ranging from 34% to 70%.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Assessment of Spin in the Abstracts of Randomized Controlled Trials in Dental Caries with Statistically Nonsignificant Results for Primary Outcomes: A Methodological Study

Su¹,

Linden²,

Faggion³

et al. 2023

Caries Res

View full text Add to dashboard Cite

The study aimed to assess the prevalence of spin in the titles and abstracts of RCTs in dental caries with statistically nonsignificant primary outcomes and to assess the risk indicators which may be associated with spin. Any original publication reporting a two-arm RCT in dental caries with clearly identified statistically nonsignificant primary outcomes published from January 1, 2015 until October 28, 2022 were included. PubMed was searched electronically to identify the eligible publications. The prevalence of spin in titles and abstracts were assessed and categorized into spin patterns based on a pre-determined classification scheme. The association between spin and the potential risk indicators at study, author, journal, institutional, and national level was assessed. A total of 234 eligible RCT publications were included. The prevalence of spin in the titles and abstracts was 3% (95%CI: 2% to 6%) and 79% (95%CI: 74% to 84%), respectively. The most common spin patterns in the results and conclusion sections, respectively, were results focusing on statistically significant within-group comparisons (23%), and conclusions focusing only on statistically significant results without acknowledgment of statistically nonsignificant results for the primary outcomes (26%). The spin was significantly associated with number of study centers (single-center vs. multicenter) (OR=2.131; 95%CI: 1.092 to 4.158; P=0.03), trial designs (non-parallel designs vs. parallel designs) (OR=0.395; 95%CI: 0.193 to 0.810; P=0.01), and overall H index of institutions for last authors (OR=0.998; 95%CI: 0.996 to 0.999; P<0.01), while it was not significantly associated with the other indicators. In the RCT publications with statistically nonsignificant results for primary outcomes in dental caries, the prevalence of spin may be low in the titles but high in the abstracts. Single-center studies with parallel designs and a lower overall H index of institutions for last authors may be more likely to have spin in the abstracts.

show abstract

Section: Interpretation Of the Main Findingsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Assessment of Spin in the Abstracts of Randomized Controlled Trials in Dental Caries with Statistically Nonsignificant Results for Primary Outcomes: A Methodological Study

Su¹,

Linden²,

Faggion³

et al. 2023

Caries Res

View full text Add to dashboard Cite

show abstract

“…And finally, recent reports indicate a high percentage (ranging from 47% to 66%) of detected spin across medical disciplines [7][8][9][10].…”

Section: Introductionmentioning

confidence: 99%

Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance

et al. 2022

View full text Add to dashboard Cite

The power of language to modify the reader’s perception of interpreting biomedical results cannot be underestimated. Misreporting and misinterpretation are pressing problems in randomized controlled trials (RCT) output. This may be partially related to the statistical significance paradigm used in clinical trials centered around a P value below 0.05 cutoff. Strict use of this P value may lead to strategies of clinical researchers to describe their clinical results with P values approaching but not reaching the threshold to be “almost significant.” The question is how phrases expressing nonsignificant results have been reported in RCTs over the past 30 years. To this end, we conducted a quantitative analysis of English full texts containing 567,758 RCTs recorded in PubMed between 1990 and 2020 (81.5% of all published RCTs in PubMed). We determined the exact presence of 505 predefined phrases denoting results that approach but do not cross the line of formal statistical significance (P < 0.05). We modeled temporal trends in phrase data with Bayesian linear regression. Evidence for temporal change was obtained through Bayes factor (BF) analysis. In a randomly sampled subset, the associated P values were manually extracted. We identified 61,741 phrases in 49,134 RCTs indicating almost significant results (8.65%; 95% confidence interval (CI): 8.58% to 8.73%). The overall prevalence of these phrases remained stable over time, with the most prevalent phrases being “marginally significant” (in 7,735 RCTs), “all but significant” (7,015), “a nonsignificant trend” (3,442), “failed to reach statistical significance” (2,578), and “a strong trend” (1,700). The strongest evidence for an increased temporal prevalence was found for “a numerical trend,” “a positive trend,” “an increasing trend,” and “nominally significant.” In contrast, the phrases “all but significant,” “approaches statistical significance,” “did not quite reach statistical significance,” “difference was apparent,” “failed to reach statistical significance,” and “not quite significant” decreased over time. In a random sampled subset of 29,000 phrases, the manually identified and corresponding 11,926 P values, 68,1% ranged between 0.05 and 0.15 (CI: 67. to 69.0; median 0.06). Our results show that RCT reports regularly contain specific phrases describing marginally nonsignificant results to report P values close to but above the dominant 0.05 cutoff. The fact that the prevalence of the phrases remained stable over time indicates that this practice of broadly interpreting P values close to a predefined threshold remains prevalent. To enhance responsible and transparent interpretation of RCT results, researchers, clinicians, reviewers, and editors may reduce the focus on formal statistical significance thresholds and stimulate reporting of P values with corresponding effect sizes and CIs and focus on the clinical relevance of the statistical difference found in RCTs.

show abstract

Spin in the Scientific Literature on Bariatric Endoscopy: a Systematic Review of Randomized Controlled Trials

et al. 2021

View full text Add to dashboard Cite

Spin occurs in bariatric surgery randomized controlled trials with a statistically nonsignificant primary outcome: A systematic review

Cited by 11 publications

References 21 publications

Assessment of Spin in the Abstracts of Randomized Controlled Trials in Dental Caries with Statistically Nonsignificant Results for Primary Outcomes: A Methodological Study

Assessment of Spin in the Abstracts of Randomized Controlled Trials in Dental Caries with Statistically Nonsignificant Results for Primary Outcomes: A Methodological Study

Analysis of 567,758 randomized controlled trials published over 30 years reveals trends in phrases used to discuss results that do not reach statistical significance

Spin in the Scientific Literature on Bariatric Endoscopy: a Systematic Review of Randomized Controlled Trials

Contact Info

Product

Resources

About