Natural Language Processing for the Identification of Surgical Site Infections in Orthopaedics

Abstract

Background: The identification of surgical site infections for infection surveillance in hospitals depends on the manual abstraction of medical records and, for research purposes, depends mainly on the use of administrative or claims data. The objective of this study was to determine whether automating the abstraction process with natural language processing (NLP)-based models that analyze the free-text notes of the medical record can identify surgical site infections with predictive abilities that match the manual abstraction process and that surpass surgical site infection identification from administrative data. Methods:We used surgical site infection surveillance data compiled by the infection prevention team to identify surgical site infections among patients undergoing orthopaedic surgical procedures at a tertiary care academic medical center from 2011 to 2017. We compiled a list of keywords suggestive of surgical site infections, and we used NLP to identify occurrences of these keywords and their grammatical variants in the free-text notes of the medical record. The key outcome was a binary indicator of whether a surgical site infection occurred. We estimated 7 incremental multivariable logistic regression models using a combination of administrative and NLP-derived variables. We split the analytic cohort into training (80%) and testing data sets (20%), and we used a tenfold cross-validation approach. The main analytic cohort included 172 surgical site infection cases and 200 controls that were repeatedly and randomly selected from a pool of 1,407 controls.Results: For Model 1 (variables from administrative data only), the sensitivity was 68% and the positive predictive value was 70%; for Model 4 (with NLP 5-grams [distinct sequences of 5 contiguous words] from the medical record), the sensitivity was 97% and the positive predictive value was 97%; and for Model 7 (a combination of Models 1 and 4), the sensitivity was 97% and the positive predictive value was 97%. Thus, NLP-based models identified 97% of surgical site infections identified by manual abstraction with high precision and 43% more surgical site infections compared with models that used administrative data only.Conclusions: Models that used NLP keywords achieved predictive abilities that were comparable with the manual abstraction process and were superior to models that used administrative data only. NLP has the potential to automate and aid accurate surgical site infection identification and, thus, play an important role in their prevention.Clinical Relevance: This study examines NLP's potential to automate the identification of surgical site infections. This automation can potentially aid the prevention and early identification of these surgical complications, thereby reducing their adverse clinical and economic impact. Surgical site infections are frequently occurring and are the most expensive of all hospital-acquired infections 1-3 . In orthopaedics, the mean surgical site infection rates range from 0.5% to 3% following hip and knee replac...

Artificial intelligence in orthopaedics: false hope or not? A narrative review along the line of Gartner’s hype cycle

“… 31 In orthopaedics, NLP has been applied to identify surgical site infections in free-text notes of medical records and achieved predictive abilities comparable with the manual abstraction process and superior to models that used administrative data only. 32 In hip arthroplasty, NLP has been used to identify common data elements 33 and classification of periprosthetic femur fractures. 34 Our group applied NLP to evaluate unstructured free-text patient-experience reviews of orthopaedic surgeons throughout the United States.…”

Validation of an algorithm for semiautomated surveillance to detect deep surgical site infections after primary total hip or knee arthroplasty—A multicenter study

“…26 These data are complex to use in automatization processes and will complicate widespread implementation. 27 Fourth, the determination of superficial SSIs requires a subjective interpretation of the definition, making them a difficult surveillance target both for manual and automated surveillance.…”

“…25,[31][32][33] In addition, coding practices differ by country, and results cannot be extrapolated. Thirukumaran et al 27 investigated the use of natural language processing in detecting SSIs. Sensitivity and PPV were extremely high in the center under study; however, the performance in other centers was not investigated, and the proposed method is rather complex to implement on a large scale compared to our method.…”

Natural Language Processing and Its Use in Orthopaedic Research

“…These results highlight that NLP is far from perfect as the topics have substantial overlap. However, four categories emerged: [ 1 ] automated reporting using electronic health record (EHR) text, [ 2 ] identifying common data elements such as infection using operative reports, [ 3 ] identifying fractures from radiology reports patient, and [ 4 ] online reviews and comments after total joint arthroplasty.…”

“…Thirukumaran et al noted that surgical site infection (SSI) is a costly problem in hospitals and is mostly tracked through manual record review of charts or administrative and claims data. They concluded that NLP was comparable to manual chart review and significantly improved SSI detection than models using administrative data alone [ 3 ]. Karhade et al also used NLP to detect reoperation rates secondary to SSI within 90 days of lumbar discectomy at two academic and three community centers.…”