Predicting Postoperative Complications after Acute Care Surgery: How Accurate is the ACS NSQIP Surgical Risk Calculator?

Abstract

The ACS NSQIP Surgical Risk Calculator (SRC) is an evidence-based clinical tool commonly used for evaluating postoperative risk. The goal of this study was to validate SRC-predicted complications by comparing them with observed outcomes in the acute care surgical setting. In this study, pre- and postoperative data from 1693 acute care surgeries (hernia repair, enterolysis, intestinal incision/excision and enterectomy, gastrectomy, debridement, colectomy, appendectomy, cholecystectomy, gastrorrhaphy, and incision and drainage of soft tissue, breast abscesses, and removal of foreign bodies) performed at a Level I trauma center over a five-year time period were abstracted. Predictions for any and serious complications were based on SRC were compared with observed outcomes using various measures of diagnostic. When evaluated as one group, the SRC had good discriminative power for predicting any and serious complications after acute care surgeries (Area Under the Curve (AUC) 0.79, 0.81). In addition, the SRC met Brier score requirements for an informative model overall. However, the predictive accuracy of the SRC varied for various procedures within the acute care patient population. For serious complications, the diagnostic measures ranged from an AUC of 0.61 and negative likelihood ratio of 0.716 for incision & drainage soft tissue to AUC of 0.91 and negative likelihood ratio of 0.064 for gastrorrhaphy. Length of stay was significantly underestimated by the SRC overall (8.56 days, P < 0.01) and for individual procedures. The SRC performs well at predicting complications after acute care surgeries overall; however, there is great variability in performance between procedure types. Further refinements in risk stratification may improve SRC predictions.

Is the ACS‐NSQIP Risk Calculator Accurate in Predicting Adverse Postoperative Outcomes in the Emergency Setting? An Italian Single‐center Preliminary Study

“…We should consider that discrimination and calibration results could substantially change between specific types of operations. For example, Golden et al [ 9 ] collected data from 1693 acute care procedures over a 5-year time period. They found that the calculator had a good discriminative power in predicting both serious and any complications rates after acute care surgeries (AUC 0.81 and 0.79, respectively) when considering the whole population as a single group.…”

“…The authors considered the ACS-NSQIP SRC as reasonably accurate in terms of estimation of postoperative death and complication in the emergency setting, with readily obtainable objective data, which can be used in the early phases of decision making and care. Others confirmed that the ACS-NSQIP SRC accurately predicted complications after emergency procedures overall, but demonstrated a great variability in performance between procedure types [ 9 ].…”

‘Taking a calculated risk’- Is American College of Surgeon’s National Surgical Quality Improvement Program universal risk calculator a valuable tool to improve quality of surgical services in Indian surgical scenario?

“…Ultimate outcome following surgical procedure is the result of interplay of so many factors which do not influence the risk calculator. 1,3,4 In the current procedural terminology (CPT) coding system which is a key input variable for the surgical procedure, it was experienced that multiple surgical procedures if contemplated in the same patient cannot be populated in the NSQIP software. Subramaniam et al in their study involving the use of ACS NSQIP calculator in head and neck oncologic conditions have eminently observed that prior radiotherapy to the area to be operated , microvascular anastomosis or chemo radiotherapy did influence the wound outcome which may cause flap necrosis , wound breakdown or wound infection altering the need of reoperation etc.…”

“…This tool is a guide to the surgeon in arriving at decision which is an inescapable component of the counselling and informed consent. 1,2,3 Several workers have published studies validating its use for a range of procedures. 4 This may be an important tool for surgical education in addition to "quality improvement" in surgical practice.…”

Three Critical Questions That Should Be Asked Before Using Prediction Models for Clinical Decision Support

“…For example, the American College of Surgeons National Surgical Quality Improvement Program surgical risk calculator 4 is reported to have generally good accuracy across a very wide range of procedures and specialties. However, other researchers 5,6 have found that this tool has low accuracy when tested for specific patients and procedures that differ from the overall sample in terms of the prevalence of inputs and outcomes.…”