Comparison of ICD-9–based, retrospective, and prospective assessments of perioperative complications: assessment of accuracy in reporting

Abstract

<jats:sec> <jats:title>Object</jats:title> <jats:p>Large studies of ICD-9–based complication and hospital-acquired condition (HAC) chart reviews have not been validated through a comparison with prospective assessments of perioperative adverse event occurrence. Retrospective chart review, while generally assumed to underreport complication occurrence, has not been subjected to prospective study. It is unclear whether ICD-9–based population studies are more accurate than retrospective reviews or are perhaps equally susceptible to bias. To determine the validity of an ICD-9–based assessment of perioperative complications, the authors compared a prospective independent evaluation of such complications with ICD-9–based HAC data in a cohort of patients who underwent spine surgery. For further comparison, a separate retrospective review of the same cohort of patients was completed as well.</jats:p></jats:sec> <jats:sec> <jats:title>Methods</jats:title> <jats:p>A prospective assessment of complications in spine surgery over a 6-month period (May to December 2008) was completed using an independent auditor and a validated definition of perioperative complications. The auditor maintained a prospective database, which included complications occurring in the initial 30 days after surgery. All medical adverse events were included in the assessment. All patients undergoing spine surgery during the study period were eligible for inclusion; the only exclusionary criterion used was the availability of the auditor for patient assessment. From the overall patient database, 100 patients were randomly extracted for further review; in these patients ICD-9–based HAC data were obtained from coder data. Separately, a retrospective assessment of complication incidence was completed using chart and electronic medical record review. The same definition of perioperative adverse events and the inclusion of medical adverse events were applied in the prospective, ICD-9–based, and retrospective assessments.</jats:p></jats:sec> <jats:sec> <jats:title>Results</jats:title> <jats:p>Ninety-two patients had adequate records for the ICD-9 assessment, whereas 98 patients had adequate chart information for retrospective review. The overall complication incidence among the groups was similar (major complications: ICD-9 17.4%, retrospective 19.4%, and prospective 22.4%; minor complications: ICD-9 43.8%, retrospective 31.6%, and prospective 42.9%). However, the ICD-9–based assessment included many minor medical events not deemed complications by the auditor. Rates of specific complications were consistently underreported in both the ICD-9 and the retrospective assessments. The ICD-9 assessment underreported infection, the need for reoperation, deep wound infection, deep venous thrombosis, and new neurological deficits (p = 0.003, p &lt; 0.0001, p &lt; 0.0001, p = 0.0025, and p = 0.04, respectively). The retrospective review underestimated incidences of infection, the need for revision, and deep wound infection (p &lt; 0.0001 for each). Only in the capture of new cardiac events was ICD-9–based reporting more accurate than prospective data accrual (p = 0.04). The most sensitive measure for the appreciation of complication occurrence was the prospective review, followed by the ICD-9–based assessment (p = 0.05).</jats:p></jats:sec> <jats:sec> <jats:title>Conclusions</jats:title> <jats:p>An ICD-9–based coding of perioperative adverse events and major complications in a cohort of spine surgery patients revealed an overall complication incidence similar to that in a prospectively executed measure. In contrast, a retrospective review underestimated complication incidence. The ICD-9–based review captured many medical events of limited clinical import, inflating the overall incidence of adverse events demonstrated by this approach. In multiple categories of major, clinically significant perioperative complications, ICD-9–based and retrospective assessments significantly underestimated complication incidence. These findings illustrate a significant potential weakness and source of inaccuracy in the use of population-based ICD-9 and retrospective complication recording.</jats:p></jats:sec>

Complications related to instrumentation in spine surgery: a prospective analysis

“…Complications and their association with an operative approach, surgical indication, and other comor- bidities have been reported in separate manuscripts. [4][5][6]27,28 However, the rate of overall complications is higher in the present series than many others that retrospectively evaluated perioperative complications after spinal fusion. The prospective design of our investigation relied less on the medical record and recall of the investigators and more on the accumulation of essential clinical details prospectively throughout the patient's hospital course; a recent review confirmed that prospective studies identified a statistically higher incidence of complications.…”

Can Surgeons Adequately Capture Adverse Events Using the Spinal Adverse Events Severity System (SAVES) and OrthoSAVES?

“…In both studies, substantial proportions of forms were not completed by the time of patient discharge, requiring the forms to be returned to the attending surgeons to be completed retrospectively. This weakens the quality of adverse-event reporting as the underlying premise is that the data collected prospectively are superior to retrospective data [6,8,42]. Our data showed similar complication rates among prospectively and retrospectively collected data.…”

Examining the effect of hospital-level factors on mortality of very low birth weight infants using multilevel modeling

“…11,12 In turn, the use of such databases for research purposes also presents some potential limitations relating to the accuracy and validity of the coding of diagnoses. [13][14][15] Other limitations of this study include the fact that, although hospital level of care and hospital volume were known for each delivery year, data on hospital services were only available for the year 2001 and were extrapolated to the other years in which individual-level data were available. Future studies should therefore focus on the more precise evaluation of these hospital services, as their presence or absence may vary over time, in addition to other potential hospital characteristics that may affect neonatal outcomes, such as the 24-h availability of obstetrical anesthesia services and in-house obstetrical services.…”