Publication and Impact of Preprints Included in the First 100 Editions of the CDC COVID-19 Science Update: Content Analysis

Abstract

Background Preprints are publicly available manuscripts posted to various servers that have not been peer reviewed. Although preprints have existed since 1961, they have gained increased popularity during the COVID-19 pandemic due to the need for immediate, relevant information. Objective The aim of this study is to evaluate the publication rate and impact of preprints included in the Centers for Disease Control and Prevention (CDC) COVID-19 Science Update and assess the performance of the COVID-19 Science Update team in selecting impactful preprints. Methods All preprints in the first 100 editions (April 1, 2020, to July 30, 2021) of the Science Update were included in the study. Preprints that were not published were categorized as “unpublished preprints.” Preprints that were subsequently published exist in 2 versions (in a peer-reviewed journal and on the original preprint server), which were analyzed separately and referred to as “peer-reviewed preprint” and “original preprint,” respectively. Time to publish was the time interval between the date on which a preprint was first posted and the date on which it was first available as a peer-reviewed article. Impact was quantified by Altmetric Attention Score and citation count for all available manuscripts on August 6, 2021. Preprints were analyzed by publication status, publication rate, preprint server, and time to publication. Results Of the 275 preprints included in the CDC COVID-19 Science Update during the study period, most came from three servers: medRxiv (n=201, 73.1%), bioRxiv (n=41, 14.9%), and SSRN (n=25, 9.1%), with 8 (2.9%) coming from other sources. Additionally, 152 (55.3%) were eventually published. The median time to publish was 2.3 (IQR 1.4-3.7). When preprints posted in the last 2.3 months were excluded (to account for the time to publish), the publication rate was 67.8%. Moreover, 76 journals published at least one preprint from the CDC COVID-19 Science Update, and 18 journals published at least three. The median Altmetric Attention Score for unpublished preprints (n=123, 44.7%) was 146 (IQR 22-552) with a median citation count of 2 (IQR 0-8); for original preprints (n=152, 55.2%), these values were 212 (IQR 22-1164) and 14 (IQR 2-40), respectively; for peer-review preprints, these values were 265 (IQR 29-1896) and 19 (IQR 3-101), respectively. Conclusions Prior studies of COVID-19 preprints found publication rates between 5.4% and 21.1%. Preprints included in the CDC COVID-19 Science Update were published at a higher rate than overall COVID-19 preprints, and those that were ultimately published were published within months and received higher attention scores than unpublished preprints. These findings indicate that the Science Update process for selecting preprints had a high fidelity in terms of their likelihood to be published and their impact. The incorporation of high-quality preprints into the CDC COVID-19 Science Update improves this activity’s capacity to inform meaningful public health decision-making.

Phase 1 of the National Institutes of Health Preprint Pilot: Testing the viability of making preprints discoverable in PubMed Central and PubMed

“…Since June 2020, much has been studied and written about the role of preprints in communicating COVID-19 research results. Otridge J et al found that “the incorporation of high-quality preprints into the CDC COVID-19 Science Update improve[d] this activity’s capacity to inform meaningful public health decision-making.” 24 Similarly, during Phase 1, NLM found incorporating preprints within the larger corpus of curated scholarly literature made available in PMC and PubMed helped NLM contextualize the research reported in preprints, linking them to similar articles in PubMed, related data, and the larger record of citation in PMC.…”

Indian COVID-19 Preprints Submissions in bioRxiv and medRxiv Preprint Servers

“…Careful scrutiny of the high increase of preprints in the domain of biomedicine, COVID-19 pandemic has consolidated the rise of using these information resources to better understand the pattern of progressing of the virus and need to develop a vaccine (Majumder & Mandl, 2020).COVID-19 Science Update reported on topics ranging from health equity, vaccine, variants, natural history, testing, etc. (Otridge et al, 2022).…”

“…Preprint server arXiv was the first server populated and popularized by physics, mathematics, and computer science community in 1991 (Vlasschaert et al, 2020). Some researchers showed that, there have been preprints since 1961 but closed in 1967 due to resistance from journals (Otridge et al, 2022). To follow the queue, life sciences community also embraced preprint server bioRxiv founded in 2013 with over 75,000 preprints as of March 2020 (Vlasschaert et al, 2020).…”

“…When the immediacy runs into decision-making process concerning health emergency posed by COVID-19 pandemic, preprints serve as an intermediary between researchers, audience, and peer-review procedure (Majumder & Mandl, 2020; Otridge et al, 2022). Even though there has been yearning from scholars to change the nomenclature preprints to “ un-refereed manuscript”, “manuscript awaiting peer-review”, “non-reviewed manuscript ” (Ravinetto et al, 2021, p. 3), this is where a decision-support tool developed by Good Publication Practice (GPP), and International Committee of Medical Journal Editors (ICMJE) becomes imperative and relevant (Mathew et al, 2022).…”

The challenge of preprints for public health

“…Efforts to curate preprint materials have been proving to be effective regarding later acceptance for publication. Preprints included in the Centers for Disease Control and Prevention (CDC) COVID-19 Science Update had a much higher publication rate -almost two thirds if sufficient time allowed 20 . However, does not adopting such measures bring preprint servers closer to the traditional publishing model?…”