When citations come before peer-review — a new normal?

The pace at which new research on COVID-19 is happening and being reported is unprecedented. By some accounts, there are now over 500 research papers on COVID-19 being published each day. The cause for this explosion in publications is primarily due to the adoption of preprinting by scientists, sharing research directly online without formal peer-review. Proponents of preprint servers have said that COVID-19, “showcases the powerful role of preprints,” or “COVID-19 is reshaping the world of bioscience publishing” while others have suggested that rather than preprinting, scientists “would be better off watching Netflix during their quarantine” or that “Some are absolutely junk science that would never get into a journal any of us heard of but yet are widely used to support theories about how to tx pts w/ deadly disease.”

Whether you’re a proponent or a skeptic, it is hard to argue that preprints have not directly influenced policy and public discourse on COVID-19. Indeed, there is even a peer-reviewed study concluding:

“Our findings suggest that, because of the speed of their release, preprints — rather than peer-reviewed literature in the same topic area — might be driving discourse related to the ongoing COVID-19 outbreak.”

In response to the impact that preprints are having on COVID-19, bioRxiv, and medRxiv, the most popular preprint servers in life sciences have appended a warning to readers, reminding them that what is posted is not peer-reviewed.

And while preprints have not been peer-reviewed, they have been screened, they do receive a digital object identifier (DOI), and are frequently cited. In this post, we utilize the unique capability of scite’s Smart Citations–citations that display the context of the citation, the location of the citation within the citing paper, and describe whether the paper provides supporting or contradicting evidence–to highlight preprints that have received supporting and/or contradicting evidence from subsequent research papers.

As an example, let’s take the COVID-19 preprint Longitudinal characteristics of lymphocyte responses and cytokine profiles in the peripheral blood of SARS-CoV-2 infected patients posted on February 22, 2020. It concludes:

“Both IL-6 and IL-10 levels showed sustained increases in the severe group compared to the mild group”

- suggesting that two specific signaling molecules are more prevalent in severe rather than mild COVID-19 disease.

Only five days later, on February 27, 2020 another preprint offering supporting evidence appears,

“In accordance with Wan S’s [7] and Liu J’s [8] study, this study also found that the levels of IL-6 and IL-10 were associated with the severity of COVID-19 pneumonia.”

What if this information was available on all preprints and peer-reviewed publications? That’s a vision we’re working hard to bring to fruition at scite by partnering with leading academic publishers. If you believe in that vision, sign-up at scite and let preprint servers and publishers you interact with know that you’d like to see Smart Citations on their articles. Below are excerpts from papers that cite the preprint of interest and support or contradict their findings.

COVID-19 preprints through the lens of scite

1. MRCA time and epidemic dynamics of the 2019 novel coronavirus

Chi Zhang, Mei Wang bioRxiv 2020.01.25.919688; DOI: https://doi.org/10.1101/2020.01.25.919688

scite report: https://scite.ai/reports/10.1101/2020.01.25.919688

2. Estimation of the asymptomatic ratio of novel coronavirus infections (COVID-19)

Hiroshi Nishiura, Tetsuro Kobayashi, Takeshi Miyama, Ayako Suzuki, Sungmok Jung, Katsuma Hayashi, Ryo Kinoshita, Yichi Yang, Baoyin Yuan, Andrei R. Akhmetzhanov, Natalie M Linton medRxiv 2020.02.03.20020248; DOI: https://doi.org/10.1101/2020.02.03.20020248 Now published in International Journal of Infectious Diseases DOI: 10.1016/j.ijid.2020.03.020

scite report: https://scite.ai/reports/10.1101/2020.02.03.20020248

3. Using predicted imports of 2019-nCoV cases to determine locations that may not be identifying all imported cases

Pablo M De Salazar, Rene Niehus, Aimee Taylor, Caroline O Buckee, Marc Lipsitch. medRxiv 2020.02.04.20020495; DOI: https://doi.org/10.1101/2020.02.04.20020495

scite report: https://scite.ai/reports/10.1101/2020.02.04.20020495

4. Evaluating the accuracy of different respiratory specimens in the laboratory diagnosis and monitoring the viral shedding of 2019-nCoV infections

Yang Yang, Minghui Yang, Chenguang Shen, Fuxiang Wang, Jing Yuan, Jinxiu Li, Mingxia Zhang, Zhaoqin Wang, Li Xing, Jinli Wei, Ling Peng, Gary Wong, Haixia Zheng, Mingfeng Liao, Kai Feng, Jianming Li, Qianting Yang, Juanjuan Zhao, Zheng Zhang, Lei Liu, Yingxia Liu. medRxiv 2020.02.11.20021493; DOI: https://doi.org/10.1101/2020.02.11.20021493

scite report: https://scite.ai/reports/10.1101/2020.02.11.20021493

5. The role of absolute humidity on transmission rates of the COVID-19 outbreak

Wei Luo, Maimuna S Majumder, Dianbo Liu, Canelle Poirier, Kenneth D Mandl, Marc Lipsitch, Mauricio Santillana. medRxiv 2020.02.12.20022467; DOI: https://doi.org/10.1101/2020.02.12.20022467

scite report: https://scite.ai/reports/10.1101/2020.02.12.20022467

6. Longitudinal characteristics of lymphocyte responses and cytokine profiles in the peripheral blood of SARS-CoV-2 infected patients

Jing Liu, Sumeng Li, Jia Liu, Boyun Liang, Xiaobei Wang, Hua Wang, Wei Li, Qiaoxia Tong, Jianhua Yi, Lei Zhao, Lijuan Xiong, Chunxia Guo, Jin Tian, Jinzhuo Luo, Jinghong Yao, Ran Pang, Hui Shen, Cheng Peng, Ting Liu, Qian Zhang, Jun Wu, Ling Xu, Sihong Lu, Baoju Wang, Zhihong Weng, Chunrong Han, Huabing Zhu, Ruxia Zhou, Helong Zhou, Xiliu Chen, Pian Ye, Bin Zhu, Shengsong He, Yongwen He, Shenghua Jie, Ping Wei, Jianao Zhang, Yinping Lu, Weixian Wang, Li Zhang, Ling Li, Fengqin Zhou, Jun Wang, Ulf Dittmer, Mengji Lu, Yu Hu, Dongliang Yang, Xin Zheng. medRxiv 2020.02.16.20023671; DOI: https://doi.org/10.1101/2020.02.16.20023671

scite report: https://scite.ai/reports/10.1101/2020.02.16.20023671

7. The incubation period of 2019-nCoV infections among travellers from Wuhan, China

Jantien A. Backer, Don Klinkenberg, Jacco Wallinga. medRxiv 2020.01.27.20018986; DOI: https://doi.org/10.1101/2020.01.27.20018986. Now published in Eurosurveillance DOI: 10.2807/1560–7917.ES.2020.25.5.2000062

scite report: https://scite.ai/reports/10.1101/2020.01.27.20018986

8. Isolation and Characterization of 2019-nCoV-like Coronavirus from Malayan Pangolins

Kangpeng Xiao, Junqiong Zhai, Yaoyu Feng, Niu Zhou, Xu Zhang, Jie-Jian Zou, Na Li, Yaqiong Guo, Xiaobing Li, Xuejuan Shen, Zhipeng Zhang, Fanfan Shu, Wanyi Huang, Yu Li, Ziding Zhang, Rui-Ai Chen, Ya-Jiang Wu, Shi-Ming Peng, Mian Huang, Wei-Jun Xie, Qin-Hui Cai, Fang-Hui Hou, Yahong Liu, Wu Chen, Lihua Xiao, Yongyi Shen bioRxiv 2020.02.17.951335; DOI: https://doi.org/10.1101/2020.02.17.951335

scite report: https://scite.ai/reports/10.1101/2020.02.17.951335

9. Single-cell RNA expression profiling of ACE2, the putative receptor of Wuhan 2019-nCov

Yu Zhao, Zixian Zhao, Yujia Wang, Yueqing Zhou, Yu Ma, Wei Zuo bioRxiv 2020.01.26.919985; DOI: https://doi.org/10.1101/2020.01.26.919985

scite report: https://scite.ai/reports/10.1101/2020.01.26.919985

10. Time-varying transmission dynamics of Novel Coronavirus Pneumonia in China

Tao Liu, Jianxiong Hu, Jianpeng Xiao, Guanhao He, Min Kang, Zuhua Rong, Lifeng Lin, Haojie Zhong, Qiong Huang, Aiping Deng, Weilin Zeng, Xiaohua Tan, Siqing Zeng, Zhihua Zhu, Jiansen Li, Dexin Gong, Donghua Wan, Shaowei Chen, Lingchuan Guo, Yan Li, Limei Sun, Wenjia Liang, Tie Song, Jianfeng He, Wenjun Ma. bioRxiv 2020.01.25.919787; DOI: https://doi.org/10.1101/2020.01.25.919787

scite report: https://scite.ai/reports/10.1101/2020.01.25.919787

11. Uncanny similarity of unique inserts in the 2019-nCoV spike protein to HIV-1 gp120 and Gag

Prashant Pradhan, Ashutosh Kumar Pandey, Akhilesh Mishra, Parul Gupta, Praveen Kumar Tripathi, Manoj Balakrishnan Menon, James Gomes, Perumal Vivekanandan, Bishwajit Kundu. bioRxiv 2020.01.30.927871; doi: https://doi.org/10.1101/2020.01.30.927871

scite report: https://scite.ai/reports/10.1101/2020.01.30.927871

12. Clinical characteristics of 2019 novel coronavirus infection in China

Wei-jie Guan, Zheng-yi Ni, Yu Hu, Wen-hua Liang, Chun-quan Ou, Jian-xing He, Lei Liu, Hong Shan, Chun-liang Lei, David SC Hui, Bin Du, Lan-juan Li, Guang Zeng, Kowk-Yung Yuen, Ru-chong Chen, Chun-li Tang, Tao Wang, Ping-yan Chen, Jie Xiang, Shi-yue Li, Jin-lin Wang, Zi-jing Liang, Yi-xiang Peng, Li Wei, Yong Liu, Ya-hua Hu, Peng Peng, Jian-ming Wang, Ji-yang Liu, Zhong Chen, Gang Li, Zhi-jian Zheng, Shao-qin Qiu, Jie Luo, Chang-jiang Ye, Shao-yong Zhu, Nan-shan Zhong medRxiv 2020.02.06.20020974; DOI: https://doi.org/10.1101/2020.02.06.20020974

scite report: https://scite.ai/reports/10.1101/2020.02.06.20020974

Related work from scite on COVID-19:

Analyzing more than 1M citations to better understand scientific research on COVID-19

Citation Network Visualization of Articles and Preprints on COVID-19