No abstract

Adoption of preprints dramatically expanded during the COVID-19 pandemic. Many have expressed concern that the risk of flawed decision-making is increased by relying on preprint data that would not survive peer review. We therefore asked how much the information presented in preprints is expected to change after review. We quantify attrition dynamics of over 1000 epidemiological estimates first reported in 100 matched preprints studying COVID-19. We find that 89% of point estimates persist through peer review. Of these, the correlation between preprint and published estimate values is extremely high at 0.99, and there is no systematic trend toward estimate inflation or deflation during review. A higher degree of data alteration during peer review, either in terms of magnitude or deletion, might be expected in papers never published because of their lower quality, which could limit the generalizability of our results. Importantly, we find that expert peer review scores of preprint quality are not related to eventual publication in a peer reviewed journal, mitigating this concern. Uncertainty is reduced somewhat, however, as authors add another 18% of data points compared to the preprint version. Confidence interval ranges also decrease by a small but statistically significant 7%. Therefore, the evidence base presented in preprints is highly stable, and where data change during review, uncertainty is expected to decrease by a small amount on average. These results lend credence to the use of preprints, as one component of the biomedical research literature, in decision-making. These results can help inform the use of preprints during the ongoing pandemic as well as future disease outbreaks.