Predicting complications of major head and neck oncological surgery: An evaluation of the ACS NSQIP surgical risk calculator

Abstract

BackgroundThe American College of Surgeons National Surgical Quality Improvement Program (ACS NSQIP) universal surgical risk calculator is an online tool intended to improve the informed consent process and surgical decision-making. The risk calculator uses a database of information from 585 hospitals to predict a patient’s risk of developing specific postoperative outcomes.MethodsPatient records at a major Canadian tertiary care referral center between July 2015 and March 2017 were reviewed for surgical cases including one of six major head and neck oncologic surgeries: total thyroidectomy, total laryngectomy, hemiglossectomy, partial glossectomy, laryngopharyngectomy, and composite resection. Preoperative information for 107 patients was entered into the risk calculator and compared to observed postoperative outcomes. Statistical analysis of the risk calculator was completed for the entire study population, for stratification by procedure, and by utilization of microvascular reconstruction. Accuracy was assessed using the ratio of predicted to observed outcomes, Receiver Operating Characteristics (ROC), Brier score, and the Wilcoxon signed–ranked test.ResultsThe risk calculator accurately predicted the incidences for 11 of 12 outcomes for patients that did not undergo free flap reconstruction (NFF group), but was less accurate for patients that underwent free flap reconstruction (FF group). Length of stay (LOS) analysis showed similar results, with predicted and observed LOS statistically different in the overall population and FF group analyses (p = 0.001 for both), but not for the NFF group analysis (p = 0.764). All outcomes in the NFF group, when analyzed for calibration, met the threshold value (Brier scores < 0.09). Risk predictions for 8 of 12, and 10 of 12 outcomes were adequately calibrated in the FF group and the overall study population, respectively. Analyses by procedure were excellent, with the risk calculator showing adequate calibration for 7 of 8 procedural categories and adequate discrimination for all calculable categories (6 of 6).ConclusionThe NSQIP-RC demonstrated efficacy for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction. The predictive value of the metric can be improved by inclusion of several factors important for risk stratification in head and neck oncology.

Optimizing classical risk scores to predict complications in head and neck surgery: a new approach

“…There is increasing recognition of the importance of a specialty-specific ACS-NSQIP and were many the studies proving that this scale is currently inadequate for head and neck surgery. Indeed, efforts to develop disease-and procedure-specific preoperative, intraoperative, and postoperative variables specific to head and neck surgery have been undertaken [8,32]. We agree with other authors who state that an essential first step in mitigating the inaccuracy of ACS-NSQIP for head and neck procedures, is the combination of CPT codes [28].…”

“…In addition, Schneider et al have added total laryngectomy to the list of complex procedures for which the NSQIP risk calculator may not be as accurate in predicting postoperative adverse events [31]. More recently, Vosler et al evaluated 131 patients and reported efficacy of ACS-NSQIP surgical calculator for predicting postoperative complications in head and neck oncology surgeries that do not require microvascular reconstruction [8]. The same authors suggest that this surgical calculator can be improved by the inclusion of several factors important for risk stratification in head and neck oncology, namely, the performance of free flap reconstruction.…”

“…Various scoring systems have been developed to estimate the risk of perioperative morbidity and mortality and these might be valuable in the selection of patients for prehabilitation before surgery. This is particularly important in the head and neck oncology discipline where complications such as fistula development can result in the significantly extended length of stay and decreased quality of life [8].…”

Impact of preoperative risk factors on outcome after gastrectomy

“…Despite limiting data regarding esophagogastric patients, the ACS NSQIP risk calculator was evaluated for a broad range of various study populations, ranging from head and neck cancer patients [21] to patients who underwent cystectomy [22]. Results of these studies were ambiguous: while the risk score demonstrated low accuracy in predicting postoperative outcomes in some cohorts [22,23], it was considered as a reliable tool in the prediction of prognosis in other study populations [21,24], suggesting that further studies are needed for validation. Furthermore, we analyzed prediction models that were developed for esophagogastric surgery such as the Cologne Risk score.…”

Preoperative Cardiac Risk Assessment

“…Several studies have questioned the accuracy of the calculator for certain urologic, abdominal, head and neck, and neurosurgical procedures, but the findings were not consistent and were confounded by the inclusion of procedurespecific variables in some studies. [13][14][15][16][17] There are several unique limitations with using NSQIP datasets to build risk-prediction models. None of the NSQIP-derived calculators have been robustly externally validated, as the currently published validation studies have had limitations as discussed above.…”