Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Abstract

Objective: Automated surveillance methods increasingly replace or support conventional (manual) surveillance; the latter is labor intensive and vulnerable to subjective interpretation. We sought to validate 2 previously developed semiautomated surveillance algorithms to identify deep surgical site infections (SSIs) in patients undergoing colorectal surgeries in Dutch hospitals. Design: Multicenter retrospective cohort study. Methods: From 4 hospitals, we selected colorectal surgery patients between 2018 and 2019 based on procedure codes, and we extracted routine care data from electronic health records. Per hospital, a classification model and a regression model were applied independently to classify patients into low- or high probability of having developed deep SSI. High-probability patients need manual SSI confirmation; low-probability records are classified as no deep SSI. Sensitivity, positive predictive value (PPV), and workload reduction were calculated compared to conventional surveillance. Results: In total, 672 colorectal surgery patients were included, of whom 28 (4.1%) developed deep SSI. Both surveillance models achieved good performance. After adaptation to clinical practice, the classification model had 100% sensitivity and PPV ranged from 11.1% to 45.8% between hospitals. The regression model had 100% sensitivity and 9.0%–14.9% PPV. With both models, <25% of records needed review to confirm SSI. The regression model requires more complex data management skills, partly due to incomplete data. Conclusions: In this independent external validation, both surveillance models performed well. The classification model is preferred above the regression model because of source-data availability and less complex data-management requirements. The next step is implementation in infection prevention practices and workflow processes.

Design and Retrospective Cohort Validation of an Algorithm for Surgical Site Infection Surveillance after Hip Replacement Surgery Using Natural Language Processing and Extreme Gradient Boosting

“…The different features included in the model have been evaluated in predictive models for other types of surgical site infection 3,17,18 , with microbiological variables being the most frequent, a finding which is corroborated by the relative importance of positive cultures in our study. NLP-based methods for SSI surveillance have been validated in various pilot studies, showing more accurate results than automated surveillance techniques that rely solely on administrative data such as coding 10 .…”

“…Secondly, our model's positive predictive value does not permit confirmation of surgical site infection on the sole basis of a positive result. As our primary objective was to reduce the proportion of medical charts to be screened manually, we did not expect to rely on the algorithm to validate positive cases, but instead chose a semi-automatic approach to SSI surveillance, as in many other studies 3,18 . Thirdly, at present, our model can only be used for SSI detection in patients undergoing hip replacement.…”

Automated extraction of standardized antibiotic resistance and prescription data from laboratory information systems and electronic health records: a narrative review

“…EHRs developers are delivering an increasing number of tools, with the aim of supporting surveillance tasks, which usually utilize structured data (Hoffman and Podgurski, 2013), yet these kinds of tools are unable to manage unstructured data (such as free text). Verberk et al (2023a) initially described and validated a semiautomated surveillance algorithm for post-operative surgical site infections in multiple hospitals. In another paper (Verberk et al, 2023b), they proposed an improvement of the algorithm, including NLP as an add-on to free text clinical notes.…”