Characterization and Reach of Orthopaedic Research Posted to Preprint Servers: Are We “Undercooking” Our Science?

Abstract

Background Although biomedical preprint servers have grown rapidly over the past several years, the harm to patient health and safety remains a major concern among several scientific communities. Despite previous studies examining the role of preprints during the Coronavirus-19 pandemic, there is limited information characterizing their impact on scientific communication in orthopaedic surgery. Questions/purposes (1) What are the characteristics (subspecialty, study design, geographic origin, and proportion of publications) of orthopaedic articles on three preprint servers? (2) What are the citation counts, abstract views, tweets, and Altmetric score per preprinted article and per corresponding publication? Methods Three of the largest preprint servers (medRxiv, bioRxiv, and Research Square) with a focus on biomedical topics were queried for all preprinted articles published between July 26, 2014, and September 1, 2021, using the following search terms: “orthopaedic,” “orthopedic,” “bone,” “cartilage,” “ligament,” “tendon,” “fracture,” “dislocation,” “hand,” “wrist,” “elbow,” “shoulder,” “spine,” “spinal,” “hip,” “knee,” “ankle,” and “foot.” Full-text articles in English related to orthopaedic surgery were included, while nonclinical studies, animal studies, duplicate studies, editorials, abstracts from conferences, and commentaries were excluded. A total of 1471 unique preprints were included and further characterized in terms of the orthopaedic subspecialty, study design, date posted, and geographic factors. Citation counts, abstract views, tweets, and Altmetric scores were collected for each preprinted article and the corresponding publication of that preprint in an accepting journal. We ascertained whether a preprinted article was published by searching title keywords and the corresponding author in three peer-reviewed article databases (PubMed, Google Scholar, and Dimensions) and confirming that the study design and research question matched. Results The number of orthopaedic preprints increased from four in 2017 to 838 in 2020. The most common orthopaedic subspecialties represented were spine, knee, and hip. From 2017 to 2020, the cumulative counts of preprinted article citations, abstract views, and Altmetric scores increased. A corresponding publication was identified in 52% (762 of 1471) of preprints. As would be expected, because preprinting is a form of redundant publication, published articles that are also preprinted saw greater abstract views, citations, and Altmetric scores on a per-article basis. Conclusion Although preprints remain an extremely small proportion of all orthopaedic research, our findings suggest that nonpeer-reviewed, preprinted orthopaedic articles are being increasingly disseminated. These preprinted articles have a smaller academic and public footprint than their published counterparts, but they still reach a substantial audience through infrequent and superficial online interactions, which are far from equivalent to the engagement facilitated by peer review. Furthermore, the sequence of preprint posting and journal submission, acceptance, and publication is unclear based on the information available on these preprint servers. Thus, it is difficult to determine whether the metrics of preprinted articles are attributable to preprinting, and studies such as the present analysis will tend to overestimate the apparent impact of preprinting. Despite the potential for preprint servers to function as a venue for thoughtful feedback on research ideas, the available metrics data for these preprinted articles do not demonstrate the meaningful engagement that is achieved by peer review in terms of the frequency or depth of audience feedback. Clinical Relevance Our findings highlight the need for safeguards to regulate research dissemination through preprint media, which has never been shown to benefit patients and should not be considered as evidence by clinicians. Clinician-scientists and researchers have the most important responsibility of protecting patients from the harm of potentially inaccurate biomedical science and therefore must prioritize patient needs first by uncovering scientific truths through the evidence-based processes of peer review, not preprinting. We recommend all journals publishing clinical research adopt the same policy as Clinical Orthopaedics and Related Research ®, The Bone & Joint Journal, The Journal of Bone and Joint Surgery, and the Journal of Orthopaedic Research, removing any papers posted to preprint servers from consideration.

CORR Insights®: Characterization and Reach of Orthopaedic Research Posted to Preprint Servers: Are We “Undercooking” Our Science?

“…Rizk et al [9] examined the risks of preprints in the orthopaedic specialty, which has been generally insulated from the worst effects because of policies put in place years ago by its leading journals. These researchers found that posting of preprints in orthopaedics, while uncommon, has increased, with potentially deleterious effects if the practice is allowed to become commonplace.…”