Publication rate and citation counts for preprints released during the COVID-19 pandemic: the good, the bad and the ugly

Abstract

Background Preprints are preliminary reports that have not been peer-reviewed. In December 2019, a novel coronavirus appeared in China, and since then, scientific production, including preprints, has drastically increased. In this study, we intend to evaluate how often preprints about COVID-19 were published in scholarly journals and cited. Methods We searched the iSearch COVID-19 portfolio to identify all preprints related to COVID-19 posted on bioRxiv, medRxiv, and Research Square from January 1, 2020, to May 31, 2020. We used a custom-designed program to obtain metadata using the Crossref public API. After that, we determined the publication rate and made comparisons based on citation counts using non-parametric methods. Also, we compared the publication rate, citation counts, and time interval from posting on a preprint server to publication in a scholarly journal among the three different preprint servers. Results Our sample included 5,061 preprints, out of which 288 were published in scholarly journals and 4,773 remained unpublished (publication rate of 5.7%). We found that articles published in scholarly journals had a significantly higher total citation count than unpublished preprints within our sample (p < 0.001), and that preprints that were eventually published had a higher citation count as preprints when compared to unpublished preprints (p < 0.001). As well, we found that published preprints had a significantly higher citation count after publication in a scholarly journal compared to as a preprint (p < 0.001). Our results also show that medRxiv had the highest publication rate, while bioRxiv had the highest citation count and shortest time interval from posting on a preprint server to publication in a scholarly journal. Conclusions We found a remarkably low publication rate for preprints within our sample, despite accelerated time to publication by multiple scholarly journals. These findings could be partially attributed to the unprecedented surge in scientific production observed during the COVID-19 pandemic, which might saturate reviewing and editing processes in scholarly journals. However, our findings show that preprints had a significantly lower scientific impact, which might suggest that some preprints have lower quality and will not be able to endure peer-reviewing processes to be published in a peer-reviewed journal.

Results availability and timeliness of registered COVID-19 clinical trials: interim cross-sectional results from the DIRECCT study

“…The low conversion of preprints to journal articles is consistent with other research on preprints during COVID-19. 40 41 The time to journal publication for preprinted trials requires additional follow-up and consideration alongside emerging evidence on the relationship between preprints and the peer-reviewed COVID-19 academic literature. 17 42 …”

A Review of the Scientific Contributions of Nepal on COVID-19

“…Globally, as the submission of research papers on COVID-19 increased exponentially, multiple scholarly journals too accelerated time to publication. [ 3 ] Several other journals issued special COVID-19 themed issues. [ 4 ] COVID-19 preprints were often shorter and were reviewed faster.…”

Unmasking Opportunities to Enhance Research Reporting and Reviewing

“…For researchers, closed laboratories freed up time to write 5 . Clinicians, who were encountering a novel virus with perhaps unique clinical phenotype, were eager to share their bedside observations and management strategies 2, 9 . Some journal editors, who were receiving a flood of manuscript submissions related to novel observations and cutting‐edge diagnostic and therapeutic strategies, were keen on publishing preprints (before peer review) of new scientific information 4 .…”

“…Some journal editors, who were receiving a flood of manuscript submissions related to novel observations and cutting‐edge diagnostic and therapeutic strategies, were keen on publishing preprints (before peer review) of new scientific information 4 . With preprints, the scientific community is expected to raise concerns about the validity of findings prior to formal publication, and thus, the advantage of disseminating research early may outweigh the disadvantage of sharing invalid findings 10 . However, an unexpected consequence emerged: journal editors retracted a large number of publications owing to questions related to the integrity of research and validity of findings, 4, 11 and many manuscripts never achieved final publication status 11 .…”

“…4 With preprints, the scientific community is expected to raise concerns about the validity of findings prior to formal publication, and thus, the advantage of disseminating research early may outweigh the disadvantage of sharing invalid findings. 10 However, an unexpected consequence emerged: journal editors retracted a large number of publications owing to questions related to the integrity of research and validity of findings, 4,11 and many manuscripts never achieved final publication status. 11 Through February 2021, the Retraction Watch online page reported at least 85 COVID-19-related manuscripts were retracted or withdrawn 13 and more preprints are expected to be withdrawn following peer review.…”