Development and Evaluation of the Universal ACS NSQIP Surgical Risk Calculator: A Decision Aid and Informed Consent Tool for Patients and Surgeons

Abstract

BACKGROUND Accurately estimating surgical risks is critical for shared decision making and informed consent. The Centers for Medicare and Medicaid Services may soon put forth a measure requiring surgeons to provide patients with patient-specific, empirically-derived estimates of postoperative complications. Our objectives were (1) to develop a universal surgical risk estimation tool, (2) to compare performance of the universal vs. prior procedure-specific Surgical Risk Calculators, and (3) to allow surgeons to empirically adjust the estimates of risk. STUDY DESIGN Using standardized clinical data from 393 ACS NSQIP hospitals, a web-based tool was developed to allow surgeons to easily enter 21 preoperative factors (demographics, comorbidities, procedure). Regression models were developed to predict 8 outcomes based on the preoperative risk factors. The universal model was compared to procedure-specific models. To incorporate surgeon input, a subjective Surgeon Adjustment Score, allowing risk estimates to vary within the estimate's confidence interval, was introduced and tested with 80 surgeons using 10 case scenarios. RESULTS Based on 1,414,006 patients encompassing 1,557 unique CPT codes, a universal Surgical Risk Calculator model was developed which had excellent performance for mortality (c-statistic=0.944; Brier=0.011[ where scores approaching zero are better]), morbidity (c-statistic=0.816, Brier=0.069), and 6 additional complications (c-statistics>0.8). Predictions were similarly robust for the universal calculator vs. procedure-specific calculators (e.g., colorectal). Surgeons demonstrated considerable agreement on the case scenario scoring (80-100% agreement), suggesting reliable score assignment between surgeons. CONCLUSIONS The ACS NSQIP Surgical Risk Calculator is a decision-support tool based on reliable multi-institutional clinical data which can be used to estimate the risks of most operations. The ACS NSQIP Surgical Risk Calculator will allow clinicians and patients to make decisions using empirically derived, patient-specific postoperative risks.

Risk Calculators Predict Failures of Knee and Hip Arthroplasties: Findings from a Large Health Maintenance Organization

“…Although risk calculators have been created for various diagnoses, procedures, and outcomes, and are used in a range of settings [2,3,5,6,16,18,22,26,35,[42][43][44][45], risk calculators for total joint arthroplasties have not been developed and fully integrated in large healthcare settings. One risk calculator for periprosthetic joint infection and mortality after THA has been developed however, it was developed only for the Medicare population [5].…”

The criteria of “inoperability”

“…Using standardized clinical data from 393 hospitals, a web-based tool was developed (http://riskcalculator.facs.org) to allow surgeons to easily enter 21 preoperative factors relating to demographics, comorbidities and procedures. Regression models were developed to predict eight outcomes based on preoperative risk factors [8]. These models are able to predict a range of postoperative events, such as death, cardiac complications, pneumonia, and acute kidney injury, with good accuracy [8].…”

“…Regression models were developed to predict eight outcomes based on preoperative risk factors [8]. These models are able to predict a range of postoperative events, such as death, cardiac complications, pneumonia, and acute kidney injury, with good accuracy [8]. While clearly the best tool, nevertheless, the ACS NSQIP risk calculator has two notable limitations: first, the prediction models were derived from a very large multicenter observational dataset that is yet to undergo external validation, especially in settings outside the United States; and, second, it incorporates the American Society of Anesthesiologists Physical Status (ASA-PS) classification system, which has limited interrater reliability [9].…”

Calidad en cirugía: hacia una mejor comprensión de las complicaciones quirúrgicas

“…Considera 21 predictores que, asociados a un procedimiento definido, puede predecir nueve posibles evoluciones dentro de los primeros 30 días de la cirugía. Es materia de recientes publicaciones 26,27 . En la actualidad, los ajustes y comparaciones en calidad se hacen fundamentalmente sobre las observaciones del NSQIP, en hospitales de Norteamérica.…”