Background
Two systems measure surgical site infection rates following colorectal surgeries. Center for Medicare and Medicaid Services pay-for-performance initiatives use National Healthcare Safety Network data for hospital comparisons.
Objective
Compare database concordance.
Design
Multi-institution cohort study of system-wide Colorectal Surgery Collaborative. The National Surgical Quality Improvement Program requires rigorous, standardized data capture techniques; National Healthcare Safety Network allows five data capture techniques. Standardized surgical site infection rates were compared between databases. Cohen's Kappa coefficient calculated.
Setting
Boston-area hospitals.
Patients
National Healthcare Safety Network or National Surgical Quality Improvement Program patients undergoing colorectal surgery.
Main Outcome Measures
Standardized surgical-site infection rates
Results
Thirty-day surgical-site infection rates of 3,547 (National Surgical Quality Improvement Program) vs 5,179 (National Healthcare Safety Network) colorectal procedures (2012-2014). Discrepancies appeared: National Surgical Quality Improvement Program database of Hospital 1 (N=1,480 patients) routinely found surgical-site infection rates around 10%, routinely deemed rate “exemplary” or “as expected” (100%). National Healthcare Safety Network data from the same hospital and time period (N=1,881) revealed similar overall surgical-site infection rate (10%), but standardized rates were deemed “worse than national average” 80% of the time. Overall, hospitals using less rigorous capture methods had improved surgical-site infection rates for National Healthcare Safety Network compared to standardized National Surgical Quality Improvement Program reports. The correlation coefficient between standardized infection rates was 0.03 (p=0.88). During 25 site-time period observations, National Surgical Quality Improvement Program and National Healthcare Safety Network data matched for 52% of observations (13/25). Kappa=0.10 (95% CI: -0.1366-0.3402; p=0.403), indicating poor agreement.
Limitations
Hospitals located in Northeastern United States only.
Conclusions
Variation in Center for Medicare and Medicaid Services-mandated National Healthcare Safety Network infection surveillance methodology leads to unreliable results, which is apparent when these results are compared to standardized data. High quality data would improve care quality and compare outcomes amongst institutions.