Factors Predicting Surgical Effort Using Explainable Artificial Intelligence in Advanced Stage Epithelial Ovarian Cancer

Abstract

(1) Background: Surgical cytoreduction for epithelial ovarian cancer (EOC) is a complex procedure. Encompassed within the performance skills to achieve surgical precision, intra-operative surgical decision-making remains a core feature. The use of eXplainable Artificial Intelligence (XAI) could potentially interpret the influence of human factors on the surgical effort for the cytoreductive outcome in question; (2) Methods: The retrospective cohort study evaluated 560 consecutive EOC patients who underwent cytoreductive surgery between January 2014 and December 2019 in a single public institution. The eXtreme Gradient Boosting (XGBoost) and Deep Neural Network (DNN) algorithms were employed to develop the predictive model, including patient- and operation-specific features, and novel features reflecting human factors in surgical heuristics. The precision, recall, F1 score, and area under curve (AUC) were compared between both training algorithms. The SHapley Additive exPlanations (SHAP) framework was used to provide global and local explainability for the predictive model; (3) Results: A surgical complexity score (SCS) cut-off value of five was calculated using a Receiver Operator Characteristic (ROC) curve, above which the probability of incomplete cytoreduction was more likely (area under the curve [AUC] = 0.644; 95% confidence interval [CI] = 0.598–0.69; sensitivity and specificity 34.1%, 86.5%, respectively; p = 0.000). The XGBoost outperformed the DNN assessment for the prediction of the above threshold surgical effort outcome (AUC = 0.77; 95% [CI] 0.69–0.85; p < 0.05 vs. AUC 0.739; 95% [CI] 0.655–0.823; p < 0.95). We identified “turning points” that demonstrated a clear preference towards above the given cut-off level of surgical effort; in consultant surgeons with <12 years of experience, age <53 years old, who, when attempting primary cytoreductive surgery, recorded the presence of ascites, an Intraoperative Mapping of Ovarian Cancer score >4, and a Peritoneal Carcinomatosis Index >7, in a surgical environment with the optimization of infrastructural support. (4) Conclusions: Using XAI, we explain how intra-operative decisions may consider human factors during EOC cytoreduction alongside factual knowledge, to maximize the magnitude of the selected trade-off in effort. XAI techniques are critical for a better understanding of Artificial Intelligence frameworks, and to enhance their incorporation in medical applications.

Artificial intelligence performance in image-based ovarian cancer identification: A systematic review and meta-analysis

“…The team also made a pioneering attempt to implement XAI models to explain the prediction of surgical effort at OC cytoreduction, by feeding the models with features that also include human factors. 84 However, most of radiomics extraction and imaging biomarkers analyses included in this review are used as “black box”, and their application in clinical practice still lacks reliability and interpretability. This phenomenon is understandable given that the use of XAI in oncology is still in its infancy.…”

Stratification of Length of Stay Prediction following Surgical Cytoreduction in Advanced High-Grade Serous Ovarian Cancer Patients Using Artificial Intelligence; the Leeds L-AI-OS Score

“…To overcome the resilience for the widest adaptation in the clinical environment, Explainability Artificial Intelligence (XAI) can be powerful to unveil the potential “black box” of AI [ 32 ]. Our team has pioneered the implementation of XAI in the EOC trajectory and provided insight into the potential influence of human factors on surgical decision-making at cytoreduction [ 33 , 34 ]. Due to some features’ collinearity, it was also inevitable that a post-operative feature was included amongst the list of features forming the post-hoc model, which would have even affected the overall model’s accuracy.…”

Beauty is in the explainable artificial intelligence (XAI) of the “agnostic” beholder

“…Arguably, no controlling mechanism against the creation of intentionally misleading interpretations exists, which can hide biases (5). Ironically, our team has recently demonstrated the potential influence of human factors on the surgical effort exerted by end-users using agnostic XAI models (6).…”