Automated surveillance systems for healthcare-associated infections: results from a European survey and experiences from real-life utilization

Verberk, Janneke D M; Aghdassi, Seven Johannes Sam; Abbas, Mohamed; Nauclér, Pontus; Gubbels, Sophie Madeleine; Maldonado, Natalia; Palacios‐Baena, Zaira R.; Johansson, Anders; Gastmeier, Petra; Behnke, Michael; Rooden, Stephanie M. van; Mourik, Maaike S. M. van

doi:10.1016/j.jhin.2021.12.021

Cited by 27 publications

(16 citation statements)

References 45 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SSI benchmarking or widespread implementation of this semiautomated algorithm may be hampered for countries without this follow-up. Last, as actual widespread implementation of automated surveillance is still limited, [24][25][26] this study provides insights into validity and data requirements needed for implementation of semiautomated SSI surveillance after colorectal surgery. However, this study did not include a full feasibility study including economic, legal, and operational assessments.…”

Section: Discussionmentioning

confidence: 99%

“…24 Hence, knowledge about generalizability of automated surveillance models is limited, and information about the path toward actual implementation is needed. 22,25,26 In this study, we present an independent and external validation of the previously developed classification and regression model in new cohorts of patients that underwent colorectal surgeries in different types of Dutch hospitals. 21 We investigated the feasibility of data requirements for both algorithms.…”

mentioning

confidence: 99%

See 1 more Smart Citation

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Verberk

Kooi

Hetem

et al. 2022

Infect. Control Hosp. Epidemiol.

View full text Add to dashboard Cite

Objective: Automated surveillance methods increasingly replace or support conventional (manual) surveillance; the latter is labor intensive and vulnerable to subjective interpretation. We sought to validate 2 previously developed semiautomated surveillance algorithms to identify deep surgical site infections (SSIs) in patients undergoing colorectal surgeries in Dutch hospitals. Design: Multicenter retrospective cohort study. Methods: From 4 hospitals, we selected colorectal surgery patients between 2018 and 2019 based on procedure codes, and we extracted routine care data from electronic health records. Per hospital, a classification model and a regression model were applied independently to classify patients into low- or high probability of having developed deep SSI. High-probability patients need manual SSI confirmation; low-probability records are classified as no deep SSI. Sensitivity, positive predictive value (PPV), and workload reduction were calculated compared to conventional surveillance. Results: In total, 672 colorectal surgery patients were included, of whom 28 (4.1%) developed deep SSI. Both surveillance models achieved good performance. After adaptation to clinical practice, the classification model had 100% sensitivity and PPV ranged from 11.1% to 45.8% between hospitals. The regression model had 100% sensitivity and 9.0%–14.9% PPV. With both models, <25% of records needed review to confirm SSI. The regression model requires more complex data management skills, partly due to incomplete data. Conclusions: In this independent external validation, both surveillance models performed well. The classification model is preferred above the regression model because of source-data availability and less complex data-management requirements. The next step is implementation in infection prevention practices and workflow processes.

show abstract

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Verberk

Kooi

Hetem

et al. 2022

Infect. Control Hosp. Epidemiol.

View full text Add to dashboard Cite

show abstract

“…Other pilot studies have been published recently using algorithm for automated or semiautomated HAI detection, combining mainly clinical and microbiological data in addition to antibiotic prescriptions. 29,30 However, performance indicators such as specificity, sensitivity, and predictive values of the algorithms have not been clearly evaluated at this time, and more surveillance data are needed. For instance, in our study, deep-incisional or organ-space SSIs are probably more easily detected than superficial SSI, because these more frequently require reoperation, although the algorithm also detects possible SSI based on positive microbiological samples with no reoperation.…”

Section: Discussionmentioning

confidence: 99%

Comorbidities directly extracted from the hospital database for adjusting SSI risk in the new national semiautomated surveillance system in France: The SPICMI network

Picard,

Nkoumazok,

Arnaud

et al. 2023

Infect. Control Hosp. Epidemiol.

View full text Add to dashboard Cite

Objective: To evaluate the performance of a comorbidity-based risk-adjustment model for surgical-site infection (SSI) reporting and benchmarking using a panel of variables extracted from the hospital discharge database (HDD), including comorbidities, compared to other models that use variables from different data sources. Methods: The French national surveillance program for SSI (SPICMI) has collected data from voluntary hospitals in the first 6 months of 2020 and 2021, for 16 selected surgery procedures, using a semiautomated algorithm for detection. Four risk-adjustment models were selected with logistic regression analysis, combining the different patterns of variables: National Nosocomial Infections Surveillance System (NNIS) risk-index components, individual operative data, and 6 individual comorbidities according to International Classification of Disease, Tenth Revision (ICD-10) diagnosis: obesity, diabetes, malnutrition, hypertension, cancer, or immunosuppression. Areas under the curve (AUCs) were calculated and compared. Results: Overall, 294 SSI were detected among 11,975 procedures included. All 6 comorbidities were related to SSI in the univariate analysis. The AUC of the selected model including comorbidities (0.675; 95% confidence interval [CI], 0.642–0.707), was significantly higher than the AUC of the model without comorbidities (0.641; 95% CI, 0.609–0.672; P = .016) or the AUC using the NNIS-index components (0.598; 95% CI, 0.564–0.630; P < .001). The HDD-based model AUC (0.659; 95% CI, 0.625–0.692) did not differ significantly from the selected model without comorbidities (P = .23). Conclusion: Including HDD-based comorbidities as patient case-mix variables instead of NNIS risk index factors could be an effective approach for risk-adjustment of automated SSI surveillance more widely accessible to hospitals.

show abstract

“…Nevertheless, validation of administrative codes to identify CIED infection lays the foundation to adapt these algorithms in clinical practice. Potential applications include fully automated or semi-automated surveillance supplemented by traditional IPC methods [ 26 ]; however, implementation will depend on local health system needs, acceptability and resources.…”

Section: Discussionmentioning

confidence: 99%

Validating administrative data to identify complex surgical site infections following cardiac implantable electronic device implantation: a comparison of traditional methods and machine learning

Rennert-May

Leal

MacDonald

et al. 2022

Antimicrob Resist Infect Control

View full text Add to dashboard Cite

Background Cardiac implantable electronic device (CIED) surgical site infections (SSIs) have been outpacing the increases in implantation of these devices. While traditional surveillance of these SSIs by infection prevention and control would likely be the most accurate, this is not practical in many centers where resources are constrained. Therefore, we explored the validity of administrative data at identifying these SSIs. Methods We used a cohort of all patients with CIED implantation in Calgary, Alberta where traditional surveillance was done for infections from Jan 1, 2013 to December 31, 2019. We used this infection subgroup as our “gold standard” and then utilized various combinations of administrative data to determine which best optimized the sensitivity and specificity at identifying infection. We evaluated six approaches to identifying CIED infection using administrative data, which included four algorithms using International Classification of Diseases codes and/or Canadian Classification of Health Intervention codes, and two machine learning models. A secondary objective of our study was to assess if machine learning techniques with training of logistic regression models would outperform our pre-selected codes. Results We determined that all of the pre-selected algorithms performed well at identifying CIED infections but the machine learning model was able to produce the optimal method of identification with an area under the receiver operating characteristic curve (AUC) of 96.8%. The best performing pre-selected algorithm yielded an AUC of 94.6%. Conclusions Our findings suggest that administrative data can be used to effectively identify CIED infections. While machine learning performed the most optimally, in centers with limited analytic capabilities a simpler algorithm of pre-selected codes also has excellent yield. This can be valuable for centers without traditional surveillance to follow trends in SSIs over time and identify when rates of infection are increasing. This can lead to enhanced interventions for prevention of SSIs.

show abstract

Automated surveillance systems for healthcare-associated infections: results from a European survey and experiences from real-life utilization

Cited by 27 publications

References 45 publications

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Semiautomated surveillance of deep surgical site infections after colorectal surgeries: A multicenter external validation of two surveillance algorithms

Comorbidities directly extracted from the hospital database for adjusting SSI risk in the new national semiautomated surveillance system in France: The SPICMI network

Validating administrative data to identify complex surgical site infections following cardiac implantable electronic device implantation: a comparison of traditional methods and machine learning

Contact Info

Product

Resources

About