Prediction of American Society of Anesthesiologists Physical Status Classification from Preoperative Clinical Text Narratives Using Natural Language Processing

Abstract

Importance: Large volumes of unstructured text notes exist for patients in electronic health records (EHR) that describe their state of health. Natural language processing (NLP) can leverage this information for perioperative risk prediction. Objective: Predict a modified American Society of Anesthesiologists Physical Status Classification (ASA-PS) score using preoperative note text, identify which model architecture and note sections are most useful, and interpret model predictions with Shapley values. Design: Retrospective cohort analysis from an EHR. Setting: Two-hospital integrated care system comprising a tertiary/quaternary academic medical center and a level 1 trauma center with a 5-state referral catchment area. Participants: Patients undergoing procedures requiring anesthesia care spanning across all procedural specialties from January 1, 2016 to March 29, 2021 who were not assigned ASA VI and also had a preoperative evaluation note filed within 90 days prior to the procedure. Exposures: Each procedural case paired with the most recent anesthesia preoperative evaluation note preceding the procedure. Main Outcomes and Measures: Prediction of a modified ASA-PS from preoperative note text. We compared 4 different text classification models for 8 different input text snippets. Performance was compared using area under the receiver operating characteristic curve (AUROC) and area under the precision recall curve (AUPRC). Shapley values were used to explain model predictions. Results: Final dataset includes 38566 patients undergoing 61503 procedures. Prevalence of ASA-PS was 8.81% for ASA I, 31.4% for ASA II, 43.25% for ASA III, and 16.54% for ASA IV-V. The best performing models were the BioClinicalBERT model on the truncated note task (macro-average AUROC 0.845) and the fastText model on the full note task (macro-average AUROC 0.865). Shapley values reveal human-interpretable model predictions. Conclusions and Relevance: Text classification models can accurately predict a patient's illness severity using only free-form text descriptions of patients without any manual data extraction. They can be an additional patient safety tool in the perioperative setting and reduce manual chart review for medical billing. Shapley feature attributions produce explanations that logically support model predictions and are understandable to clinicians.

SADA: enhanced framework for predicting surgical risk via adaptive data augmentation and high-risk identification

“…Michael et al [15] constructed a series of machine learning models using 12064 patients, including gradient boosting, logistic regression, linear discriminant analysis, AdaBoost, and support vector machines (SVM), enabling objective and reproducible scoring of ASA-PS. In addition, related work includes [16] [17]. Different from the above methods, we focus on solving the problems of unbalanced medical data and difficulty in identifying high-risk surgical samples, which still need to be solved in the above papers, to further improve the prediction performance of surgical risk based on the prediction of ASA-PS.…”