Objective: To characterize agreement between administrative and registry data in the determination of patient-level comorbidities. Background: Previous research finds poor agreement between these 2 types of data in the determination of outcomes. We hypothesized that concordance between administrative and registry data would also be poor. Methods: A cohort of inpatient operations (length of stay 1 day or greater) was obtained from a consortium of 8 hospitals. Within each hospital, National Surgical Quality Improvement Program (NSQIP) data were merged with intra-institutional inpatient administrative data. Twelve different comorbidities (diabetes, hypertension, congestive heart failure, hemodialysis-dependence, cancer diagnosis, chronic obstructive pulmonary disease, ascites, sepsis, smoking, steroid, congestive heart failure, acute renal failure, and dyspnea) were analyzed in terms of agreement between administrative and NSQIP data. Results: Forty-one thousand four hundred thirty-two inpatient surgical hospitalizations were analyzed in this study. Concordance (Cohen Kappa value) between the 2 data sources varied from 0.79 (diabetes) to 0.02 (dyspnea). Hospital variation in concordance (intersite variation) was quantified using a test of homogeneity. This test found significant intersite variation at a level of P < 0.001 for each of the comorbidities except for dialysis (P = 0.07) and acute renal failure (P = 0.19). These findings imply significant differences between hospitals in their generation of comorbidity data. Conclusion: This study finds significant differences in how administrative versus registry data assess patient-level comorbidity. These differences are of concern to patients, payers, and providers, each of which had a stake in the integrity of these data. Standardized definitions of comorbidity and periodic audits are necessary to ensure data accuracy and minimize bias.

With a goal of improving existing mechanisms for measuring surgical quality, definitions for the occurrence of a postoperative complication need to be developed and applied consistently. Progress toward this goal will enable patients and payers to better take advantage of recent advances in healthcare data transparency.

This study identifies significant problems in ability of both NSQIP and administrative data to assess postoperative VT/PE. Administrative data functioned more accurately than NSQIP data in the identification of postoperative PE. The mechanisms used to translate VTE measurement into quality improvement should be standardized and improved.