Development and internal validation of a machine-learning-developed model for predicting 1-year mortality after fragility hip fracture

Abstract

Background Fragility hip fracture increases morbidity and mortality in older adult patients, especially within the first year. Identification of patients at high risk of death facilitates modification of associated perioperative factors that can reduce mortality. Various machine learning algorithms have been developed and are widely used in healthcare research, particularly for mortality prediction. This study aimed to develop and internally validate 7 machine learning models to predict 1-year mortality after fragility hip fracture. Methods This retrospective study included patients with fragility hip fractures from a single center (Siriraj Hospital, Bangkok, Thailand) from July 2016 to October 2018. A total of 492 patients were enrolled. They were randomly categorized into a training group (344 cases, 70%) or a testing group (148 cases, 30%). Various machine learning techniques were used: the Gradient Boosting Classifier (GB), Random Forests Classifier (RF), Artificial Neural Network Classifier (ANN), Logistic Regression Classifier (LR), Naive Bayes Classifier (NB), Support Vector Machine Classifier (SVM), and K-Nearest Neighbors Classifier (KNN). All models were internally validated by evaluating their performance and the area under a receiver operating characteristic curve (AUC). Results For the testing dataset, the accuracies were GB model = 0.93, RF model = 0.95, ANN model = 0.94, LR model = 0.91, NB model = 0.89, SVM model = 0.90, and KNN model = 0.90. All models achieved high AUCs that ranged between 0.81 and 0.99. The RF model also provided a negative predictive value of 0.96, a positive predictive value of 0.93, a specificity of 0.99, and a sensitivity of 0.68. Conclusions Our machine learning approach facilitated the successful development of an accurate model to predict 1-year mortality after fragility hip fracture. Several machine learning algorithms (eg, Gradient Boosting and Random Forest) had the potential to provide high predictive performance based on the clinical parameters of each patient. The web application is available at www.hipprediction.com. External validation in a larger group of patients or in different hospital settings is warranted to evaluate the clinical utility of this tool. Trial registration Thai Clinical Trials Registry (22 February 2021; reg. no. TCTR20210222003).

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction

“…Of 39 studies that met all criteria and were included in this analysis, 18 studies (46.2%) used AI models to diagnose hip fractures on plain radiographs and 21 studies (53.8%) used AI models to predict patient outcomes following hip fracture surgery. A PRISMA flowchart of included studies is displayed in eFigure 1 in Supplement 1.…”

“…Machine learning models have been developed to predict the outcome of 6 different postoperative outcomes following hip fracture surgery: mortality (15 studies), length of stay (3 studies), delirium (1 study), discharge destination (1 study), hospital cost (1 study), 30-day major complications (1 study), and functional independence measure (1 study) (Table 2). Age (18 of 21 studies [85.7%]) and sex (17 of 21 studies [80.9%]) were the most used features, whereas all other input features varied widely across studies and databases (eTable 4 in Supplement 1).…”

“…In this study, the meta-analysis conducted to compare the accuracy of these ML models revealed that the models are comparable with the mean performance of expert clinicians at diagnosing hip fractures. Across all included studies, there was a wider range of sensitivity and specificity compared with clinician performance. However, the range was negatively skewed by 1 study that attempted to classify fractures into 3 different categories despite a relatively small training sample size.…”

“…Included Studies on Application of Artificial Intelligence for Diagnosis of Hip FracturesArtificial Intelligence for Hip Fracture Detection and Outcome Prediction Age (18 of 21 studies47,[49][50][51][52][54][55][56][57][58][59][60][61][63][64][65][66][67] [85.7%]) and sex (17 of 21 studies [80.9%][47][48][49][50][51][52][54][55][56][57][58]60,61,[64][65][66][67] …”

Prediction of Complications and Prognostication in Perioperative Medicine: A Systematic Review and PROBAST Assessment of Machine Learning Tools

“…Elective noncardiac surgery Mortality 30 days and/or 1 yr [26][27][28][29][30][31][32][33][34][35][36][37][38] In surgical patients with perioperative SarS-CoV-2 39 Morbidity Multiple postoperative complications 26,27,29,[40][41][42][43][44][45][46][47][48][49][50][51] acute and chronic pain 52-57 acute kidney failure 52,58-63 aSa score prediction 64 Delirium and cognitive decline [65][66][67][68][69][70] Cerebral/myocardial infarction 71 Difficult intubation prediciton 72 Ileus 73 Infection risk [74][75][76] Myocardial injury 77 Nausea and vomiting 78 Obstructive apnoea screening 79 Perioperative transfusion 80,81 Postoperative atrial fibrillation 82 respiratory failure and depression Liver failure 117 Major bleeding 118,119 Kidney failure…”