Predicting causal citations without full text

Abstract

Citation analysis generally assumes that each citation documents causal knowledge transfer that informed the conception, design, or execution of the main experiments. Citations may exist for other reasons. In this paper we identify a subset of citations that are unlikely to represent causal knowledge flow. Using a large, comprehensive feature set of open access data, we train a predictive model to identify such citations. The model relies only on the title, abstract, and reference set and not the full-text or future citations patterns, making it suitable for publications as soon as they are released, or those behind a paywall. We find that the model identifies, with high prediction scores, citations that were likely added during the peer review process, and conversely identifies with low prediction scores citations that are known to represent causal knowledge transfer. Using the model, we find that federally funded biomedical research publications represent 30% of the estimated causal knowledge transfer from basic studies to clinical research, even though these comprise only 10% of the literature, a three-fold overrepresentation in this important type of knowledge transfer. This finding underscores the importance of federal funding as a policy lever to improve human health.

Robustness of evidence reported in preprints during peer review

“… 23 Metascience studies suggest that the discrepancy between preprints and peer-reviewed articles is small and the quality of reporting is within comparable range, supporting the validity of communicating research findings in preprints before review. 24 , 25 The results of our study and others 24 , 25 , 26 , 27 , 28 , 29 , 30 suggest that the reliability of data reported in preprints is generally high. Although there are measurable effects on research articles after peer review, such as the observed reduction in CIs, effect sizes are small.…”