Validating administrative data to identify complex surgical site infections following cardiac implantable electronic device implantation: a comparison of traditional methods and machine learning

Abstract

Background Cardiac implantable electronic device (CIED) surgical site infections (SSIs) have been outpacing the increases in implantation of these devices. While traditional surveillance of these SSIs by infection prevention and control would likely be the most accurate, this is not practical in many centers where resources are constrained. Therefore, we explored the validity of administrative data at identifying these SSIs. Methods We used a cohort of all patients with CIED implantation in Calgary, Alberta where traditional surveillance was done for infections from Jan 1, 2013 to December 31, 2019. We used this infection subgroup as our “gold standard” and then utilized various combinations of administrative data to determine which best optimized the sensitivity and specificity at identifying infection. We evaluated six approaches to identifying CIED infection using administrative data, which included four algorithms using International Classification of Diseases codes and/or Canadian Classification of Health Intervention codes, and two machine learning models. A secondary objective of our study was to assess if machine learning techniques with training of logistic regression models would outperform our pre-selected codes. Results We determined that all of the pre-selected algorithms performed well at identifying CIED infections but the machine learning model was able to produce the optimal method of identification with an area under the receiver operating characteristic curve (AUC) of 96.8%. The best performing pre-selected algorithm yielded an AUC of 94.6%. Conclusions Our findings suggest that administrative data can be used to effectively identify CIED infections. While machine learning performed the most optimally, in centers with limited analytic capabilities a simpler algorithm of pre-selected codes also has excellent yield. This can be valuable for centers without traditional surveillance to follow trends in SSIs over time and identify when rates of infection are increasing. This can lead to enhanced interventions for prevention of SSIs.

Complex cardiac implantable electronic device infections in Alberta, Canada: An epidemiologic cohort study of validated administrative data

“…We linked Paceart data to the AHS DAD hospital admissions data and International Classification of Diseases, Tenth Revision in Canada (ICD-10-CA) codes to determine which patients developed a complex SSI within 1 year of the index procedure. This validated administrative algorithm, which demonstrated 91% sensitivity and 99% specificity, searched by ICD-10-CA codes 11 for infection of an implantable cardiovascular or other device (T827, T857), infective endocarditis (I330, I339, I38, I398), and cellulitis of the chest wall or other unspecified site (L0330, L0339, L038, L039). The validated algorithm was based on definitions of complex CIED infections according to the Centre for Disease Control National Healthcare Safety Network (CDC/NHSN) protocols for surgical site infection (SSI), where 'complex' SSI included both deep incisional SSI of the fascia or muscle (excluding superficial SSI of skin), and organ-space SSI.…”

“…Perhaps most importantly, our method of identifying infections using administrative data was recently validated through a comparative analysis of established ICD-10-CA algorithms and was proven to have comparable validity to the reference standard of IPC surveillance methods. 11 This now-validated method of identifying CIED infections can act as a framework for future surveillance and research.…”

“…An alternate method of infection surveillance is enabled by administrative data, which facilitates identification of CIED infection within health systems that do not routinely collect formal IPC surveillance data. 11 The use of administrative data codes for identification of complex CIED infection was recently validated with 91% sensitivity and 99% specificity. 11 These administrative codes have not previously been applied to assess the epidemiology of CIED infection following validation, particularly in the Canadian setting.…”

“…11 The use of administrative data codes for identification of complex CIED infection was recently validated with 91% sensitivity and 99% specificity. 11 These administrative codes have not previously been applied to assess the epidemiology of CIED infection following validation, particularly in the Canadian setting.…”