Tracking changes between preprint posting and journal publication during a pandemic

Abstract

Amid the Coronavirus Disease 2019 (COVID-19) pandemic, preprints in the biomedical sciences are being posted and accessed at unprecedented rates, drawing widespread attention from the general public, press, and policymakers for the first time. This phenomenon has sharpened long-standing questions about the reliability of information shared prior to journal peer review. Does the information shared in preprints typically withstand the scrutiny of peer review, or are conclusions likely to change in the version of record? We assessed preprints from bioRxiv and medRxiv that had been posted and subsequently published in a journal through April 30, 2020, representing the initial phase of the pandemic response. We utilised a combination of automatic and manual annotations to quantify how an article changed between the preprinted and published version. We found that the total number of figure panels and tables changed little between preprint and published articles. Moreover, the conclusions of 7.2% of non-COVID-19–related and 17.2% of COVID-19–related abstracts undergo a discrete change by the time of publication, but the majority of these changes do not qualitatively change the conclusions of the paper.

Correction: Prognostic factors for severity and mortality in patients infected with COVID-19: A systematic review

“…The authors have not undertaken a full assessment of the current publication status of the included preprints, and furthermore did not conduct subgroup analyses based on publication status. However recent studies examining cohorts of COVID-19 preprints vs. articles suggested that conclusions were not widely changed following peer review [ 4 , 5 ].…”

Open Science and COVID-19 Randomized Controlled Trials: Examining Open Access, Preprinting, and Data Sharing-Related Practices During the Pandemic

“…However, preprints may also be a source for questionable, fraudulent, or even just preliminary claims that can be taken up by individuals or outlets who are unaware that the source has not gone through the traditional peer-review process (Watson, 2022). Of course, questionable and fraudulent claims can also be present in peer reviewed literature ((Ledford & Van Noorden, 2020)and, while substantial changes can occur between a given paper’s preprint and peer-reviewed version, the majority of these changes do not appear to qualitatively affect the paper’s conclusion (Brierley et al, 2022).…”

Downstream retraction of preprinted research in the life and medical sciences

“…However, because it is unusual for journals to acknowledge the existence of a preprint associated with an article, information about a retraction does not reach the preprint server unless the author updates the server or there are procedures in place at the preprint server to explicitly search for such information. There is some data to suggest that there are generally few meaningful differences between preprints and their corollary journal-published articles [ 8 ]. Thus, problems discovered in the latter are likely to impact the former.…”