Automated Detection of Healthcare Associated Infections: External Validation and Updating of a Model for Surveillance of Drain-Related Meningitis

Abstract

ObjectiveAutomated surveillance of healthcare-associated infections can improve efficiency and reliability of surveillance. The aim was to validate and update a previously developed multivariable prediction model for the detection of drain-related meningitis (DRM).DesignRetrospective cohort study using traditional surveillance by infection control professionals as reference standard.PatientsPatients receiving an external cerebrospinal fluid drain, either ventricular (EVD) or lumbar (ELD) in a tertiary medical care center. Children, patients with simultaneous drains, <1 day of follow-up or pre-existing meningitis were excluded leaving 105 patients in validation set (2010–2011) and 653 in updating set (2004–2011).MethodsFor validation, the original model was applied. Discrimination, classification and calibration were assessed. For updating, data from all available years was used to optimally re-estimate coefficients and determine whether extension with new predictors is necessary. The updated model was validated and adjusted for optimism (overfitting) using bootstrapping techniques.ResultsIn model validation, the rate of DRM was 17.4/1000 days at risk. All cases were detected by the model. The area under the ROC curve was 0.951. The positive predictive value was 58.8% (95% CI 40.7–75.4) and calibration was good. The revised model also includes Gram stain results. Area under the ROC curve after correction for optimism was 0.963 (95% CI 0.953– 0.974). Group-level prediction was adequate.ConclusionsThe previously developed multivariable prediction model maintains discriminatory power and calibration in an independent patient population. The updated model incorporates all available data and performs well, also after elaborate adjustment for optimism.

Development of Infection Control Surveillance System for Intensive Care Unit: Data Requirements and Guidelines

“…Probability estimat ions through clinical parameter modelling needs to avoid the rapidly raised infection rates that are observable through infection detection algorith mically [19]. Instead of binary algorith mic grouping, total rates would be p roduced through summing of estimated probabilities [20].…”

Validation of an Automated Surveillance Approach for Drain-Related Meningitis: A Multicenter Study

“…14,18,19 In short, patients must either have microbiological growth from CSF cultures or a combination of clinical signs and anomalities in CSF consistent with meningitis and administration of antibiotic therapy directed at DRM. Importantly, cultures with common skin contaminants in the absence of clinical signs reflective of meningitis are not considered cases of DRM, and infections must occur within 7 days of drain removal to be attributed to the EVD or ELD.…”

“…Using drain-related meningitis (DRM) as a prototype infection, a system based on a multivariable regression model was recently developed that demonstrated a favorable balance between sensitivity of DRM detection and workload reduction. 2,14 Drain-related meningitis, or ventriculitis or ventriculostomyrelated infection, is an infection of the meninges and/or ventricles complicating the use of external cerebrospinal fluid (CSF) drains, in particular ventricular (EVD) and lumbar (ELD) drains. [15][16][17][18] These drains are placed to manage increased intracranial pressure, for example, due to intracerebral hemorrhage or trauma, prophylactically during surgery, or as treatment for cerebrospinal fluid leakage.…”

“…14 Prediction models require validation in both time and place to establish their transportability to other settings and in this study, we present a prospective and external validation of the multivariable automated surveillance approach to DRM in comparison to traditional manual methods. 20,21 In addition, we used this approach to validate the merits of the multivariable approach in relation to an alternative automated surveillance strategy based on a classification algorithm and to evaluate whether differences in performance can be linked to hospital or patient characteristics.…”

“…1, Appendix Table 1). 14,23 In brief, the multivariable logistic regression model uses data on microbiological culture results, antibiotic dispensing, and clinical chemistry results combined with drain characteristics to estimate a predicted probability of DRM for each patient. Missing values are handled by multiple imputation.…”