Citations and publication rate of preprints on pharmacological interventions for COVID-19: The good, the bad and, the ugly.

Abstract

Background: Preprints are preliminary reports that have not been peer-reviewed. On December 2019, a novel coronavirus appeared in China, and since then, scientific production, including preprints, has drastically increased. In this study, we intend to evaluate how often preprints regarding pharmacological interventions against COVID-19 were cited, in spite of the fact that some of these preprints remained unpublished.Methods: We conducted a search on medRxiv and bioRxiv to identify preprints related to pharmacological interventions against SARS-CoV-2 from January 1, 2020 to March 31, 2020. We included any study type that addressed or reported data on pharmacological interventions. We gathered metadata on June 26, 2020 of included preprints and identified if they had been published in a scholarly journal. We performed Mann-Whitney U tests to evaluate if published articles had differences in citation counts or metrics, as defined by PDF downloads and abstract reads, when compared to unpublished preprints.Results: Our sample included 97 preprints, of which 23 were published on scholarly journals and 74 remained unpublished (Publication rate of 23,7%). The most common study designs we found among preprints were basic science research and case series. The number of citations in our sample ranged from 0 to 1409 for published articles, and ranged from 0 to 175 citations for unpublished preprints. Published articles had a significantly higher number of citations when compared to unpublished preprints (p=0,000013). We did not find a statistical difference in PDF download (p=0,167) and abstract reads (p= 0,181). In the published articles, the time from posting on a preprint server to publication on a journal ranged from 0 to 98 days (median: 42.0 days). The time period from date of submission to a journal to date of acceptance in our sample ranged from 1 to 228 days (median: 23 days). Almost half of the preprints that were subsequently published (47,8%) had modifications made to the result section after peer-review.Conclusions: The publication rate of the preprints in this sample was low (1 in 4), although review times in scholarly journals seems to be accelerated. However, there was no difference in the number of views or downloads between preprints already published in scholarly journals and those not yet.

Cross-sectional study of preprints and final journal publications from COVID-19 studies: discrepancies in results reporting and spin in interpretation

“…Among these, almost half (11/23, 48%) had modifications in the title or results section, although the nature of these modifications is not described. 10 An analysis of spin in preprints and journal publications for COVID-19 trials found a single difference between two matched pairs of preprints and their journal publications: the discussion of limitations in the abstract. Limitations were discussed in the abstract of one article, but not in its accompanying preprint.…”

Using an expert survey and user feedback to construct PRECHECK: A checklist to evaluate preprints on COVID-19 and beyond

“…This finding is reinforced by comparisons between preprints and publications on metrics like the basic reproduction number, transmissibility, incubation period, and infectious period, showing no evidence for a difference in the results presented in peer-reviewed vs. non-peer-reviewed literature (33,34). Nonetheless, a large portion of COVID-19 preprints show substantial changes in methods and results after peer-review (nearly half of the preprints analysed by Oikonomidi (2020) and Nicolalde et al (2020) (25,27)), suggesting flaws in the most essential elements of many COVID-19 preprints. At least two potential solutions for distinguishing high-from low-quality research in preprints are possible: 1) introducing quality control measures, and 2) educating the readership of preprints to make quality evaluations themselves.…”

“…While it is true that many preprints never convert into publications (70-80% as of April 2021, see e.g., 25,26) and that unpublished preprints tend to be less cited than published ones (27,28), this is not necessarily due to poor quality. For one, the link between a preprint and its publication may be lost when it is the preprint that gets cited (29).…”

“…Alternatively, the authors' decisions could be the cause, as some may avoid the publication process altogether (30). Others may intentionally use preprints to release replications, and null results that are difficult to publish (27), or works in progress which may be less well-written and inadequate at sharing data/code (17)(18)(19)(20). Many preprints actually report their results in a balanced way so as not to 'oversell' their findings (31).…”

Comparison of preprints and final journal publications from COVID-19 Studies: Discrepancies in results reporting and spin in interpretation

“…Among these, almost half (11/23, 48%) had modifications in the title or results section, although the nature of these modifications is not described. [10] An analysis of spin in preprints and journal publications for COVID-19 trials found a single difference between 2 matched pairs preprint and their journal publications: the discussion of limitations in the abstract. Limitations were discussed in the abstract of one article, but not its accompanying preprint.…”