What difference do retractions make? An estimate of the epistemic impact of retractions on recent meta-analyses

Abstract

Every year, several hundred publications are retracted due to fabrication and falsification of data or plagiarism and other breeches of research integrity and ethics. Despite considerable research on this phenomenon, the extent to which a retraction requires revising previous scientific estimates and beliefs – which we define as the epistemic impact - is unknown. We collected a representative sample of recently retracted studies that had been included in recent meta-analyses, and compared the summary effect size of these meta analyses with and without the refracted studies. On average, the retractions had occurred about six years prior to the publication of the corresponding meta-analyses.Our results suggest that retractions have varying impacts depending on their causes. In particular, removing from an analysis a study retracted because of issues with data, methods or results, led to a statistically significant reduction of the estimated effect size. Assuming that the results of these retracted studies are completely false, then the meta-analyses that had included them had overestimated the summary effect sizes by, averaging across effect size metrics, 30% (median, 13%). However, retractions due to plagiarism or other issues not related to data, methods or results had no impact on the conclusions of meta-analyses.Since retractions due to plagiarism or other non-data related issues typically constitute over 75% of total retractions, our results suggest that the epistemic impact of most retractions is likely to be null. However, our results also suggest that retractions due to issues with data, methods or results should be accompanied by a revision of relevant meta-analyses, and by extension a downwards revision of prior scientific beliefs.

An investigation into the impact and implications of published papers from retracted research: systematic search of affected literature

“…In the case of Fujii’s extensive publications, the effect of his misconduct on the management of postoperative nausea and vomiting appears to have only been minimised by the large volume of publications from other authors 79. In a recent paper, analyses by Fanelli and Moher92 suggested that meta-analyses may overestimate their summary effect sizes when they include studies later retracted for issues with data, methods or results.…”

Media and social media attention to retracted articles according to Altmetric

“…These results align well with literature on case studies [7,8], specific disciplines [9] and the broader literature [10], which identify that at least 80% of retracted articles receive positive post-retraction citations. Such perpetuated misinformation is not inconsequential: guidelines and meta-analyses seem to be very rarely updated to remove retracted articles [11] and a recent preprint suggests that doing so would lead to a median reduction in estimated effect size of 13% and an average reduction of 30% [12].…”

Plagiarism in Brazil: A perspective of 25,000 PhD holders across the sciences

“…The retraction was for misappropriation of ideas and text from a paragraph of a manuscript that was undergoing review ( 50, 51 ). The case is provocative, as the “epistemic impact” ( 52 ) of the retraction seems minor. But the problem has broad ethical dimensions framing the relationship among authors, reviewers and editors ( 51 ).…”