Development and internal validation of a clinical prediction model using machine learning algorithms for 90 day and 2 year mortality in femoral neck fracture patients aged 65 years or above

Oosterhoff, Jacobien H F; Savelberg, Angelique Berit Marte Corlijn; Karhade, Aditya V.; Gravesteijn, Benjamin; Doornberg, Job N.; Schwab, Joseph H.; Heng, Marilyn

doi:10.1007/s00068-022-01981-4

Cited by 17 publications

(17 citation statements)

References 58 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Of 39 studies that met all criteria and were included in this analysis, 18 studies (46.2%) used AI models to diagnose hip fractures on plain radiographs and 21 studies (53.8%) used AI models to predict patient outcomes following hip fracture surgery. A PRISMA flowchart of included studies is displayed in eFigure 1 in Supplement 1.…”

Section: Resultsmentioning

confidence: 99%

“…Machine learning models have been developed to predict the outcome of 6 different postoperative outcomes following hip fracture surgery: mortality (15 studies), length of stay (3 studies), delirium (1 study), discharge destination (1 study), hospital cost (1 study), 30-day major complications (1 study), and functional independence measure (1 study) (Table 2). Age (18 of 21 studies [85.7%]) and sex (17 of 21 studies [80.9%]) were the most used features, whereas all other input features varied widely across studies and databases (eTable 4 in Supplement 1).…”

Section: Resultsmentioning

confidence: 99%

“…In this study, the meta-analysis conducted to compare the accuracy of these ML models revealed that the models are comparable with the mean performance of expert clinicians at diagnosing hip fractures. Across all included studies, there was a wider range of sensitivity and specificity compared with clinician performance. However, the range was negatively skewed by 1 study that attempted to classify fractures into 3 different categories despite a relatively small training sample size.…”

Section: Discussionmentioning

confidence: 99%

“…Included Studies on Application of Artificial Intelligence for Diagnosis of Hip FracturesArtificial Intelligence for Hip Fracture Detection and Outcome Prediction Age (18 of 21 studies47,[49][50][51][52][54][55][56][57][58][59][60][61][63][64][65][66][67] [85.7%]) and sex (17 of 21 studies [80.9%][47][48][49][50][51][52][54][55][56][57][58]60,61,[64][65][66][67] …”

mentioning

confidence: 99%

See 3 more Smart Citations

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction

et al. 2023

View full text Add to dashboard Cite

ImportanceArtificial intelligence (AI) enables powerful models for establishment of clinical diagnostic and prognostic tools for hip fractures; however the performance and potential impact of these newly developed algorithms are currently unknown.ObjectiveTo evaluate the performance of AI algorithms designed to diagnose hip fractures on radiographs and predict postoperative clinical outcomes following hip fracture surgery relative to current practices.Data SourcesA systematic review of the literature was performed using the MEDLINE, Embase, and Cochrane Library databases for all articles published from database inception to January 23, 2023. A manual reference search of included articles was also undertaken to identify any additional relevant articles.Study SelectionStudies developing machine learning (ML) models for the diagnosis of hip fractures from hip or pelvic radiographs or to predict any postoperative patient outcome following hip fracture surgery were included.Data Extraction and SynthesisThis study followed the Preferred Reporting Items for Systematic Reviews and Meta-analyses and was registered with PROSPERO. Eligible full-text articles were evaluated and relevant data extracted independently using a template data extraction form. For studies that predicted postoperative outcomes, the performance of traditional predictive statistical models, either multivariable logistic or linear regression, was recorded and compared with the performance of the best ML model on the same out-of-sample data set.Main Outcomes and MeasuresDiagnostic accuracy of AI models was compared with the diagnostic accuracy of expert clinicians using odds ratios (ORs) with 95% CIs. Areas under the curve for postoperative outcome prediction between traditional statistical models (multivariable linear or logistic regression) and ML models were compared.ResultsOf 39 studies that met all criteria and were included in this analysis, 18 (46.2%) used AI models to diagnose hip fractures on plain radiographs and 21 (53.8%) used AI models to predict patient outcomes following hip fracture surgery. A total of 39 598 plain radiographs and 714 939 hip fractures were used for training, validating, and testing ML models specific to diagnosis and postoperative outcome prediction, respectively. Mortality and length of hospital stay were the most predicted outcomes. On pooled data analysis, compared with clinicians, the OR for diagnostic error of ML models was 0.79 (95% CI, 0.48-1.31; P = .36; I2 = 60%) for hip fracture radiographs. For the ML models, the mean (SD) sensitivity was 89.3% (8.5%), specificity was 87.5% (9.9%), and F1 score was 0.90 (0.06). The mean area under the curve for mortality prediction was 0.84 with ML models compared with 0.79 for alternative controls (P = .09).Conclusions and RelevanceThe findings of this systematic review and meta-analysis suggest that the potential applications of AI to aid with diagnosis from hip radiographs are promising. The performance of AI in diagnosing hip fractures was comparable with that of expert radiologists and surgeons. However, current implementations of AI for outcome prediction do not seem to provide substantial benefit over traditional multivariable predictive statistics.

show abstract

Section: Resultsmentioning

confidence: 99%

Section: Resultsmentioning

confidence: 99%

Section: Discussionmentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction

et al. 2023

View full text Add to dashboard Cite

show abstract

“…We found that our c-statistics were comparable for models developed for 30-day and 90-day mortality prediction showing c-statistics ranging from 0.73 to 0.92. 24,26,27,29,63 However, most studies did not report calibration metrics or Brier scores. [24][25][26][27][28][29] This study identified the following predictive variables for 90-day mortality: pre-fracture functional status, ASA grade, sex, osteoarthritis, age, anaemia or other blood disease, pre-fracture living status, ethnicity, and BMI.…”

Section: Figmentioning

confidence: 99%

Development of machine-learning algorithms for 90-day and one-year mortality prediction in the elderly with femoral neck fractures based on the HEALTH and FAITH trials

et al. 2023

View full text Add to dashboard Cite

AimsTo develop prediction models using machine-learning (ML) algorithms for 90-day and one-year mortality prediction in femoral neck fracture (FNF) patients aged 50 years or older based on the Hip fracture Evaluation with Alternatives of Total Hip arthroplasty versus Hemiarthroplasty (HEALTH) and Fixation using Alternative Implants for the Treatment of Hip fractures (FAITH) trials.MethodsThis study included 2,388 patients from the HEALTH and FAITH trials, with 90-day and one-year mortality proportions of 3.0% (71/2,388) and 6.4% (153/2,388), respectively. The mean age was 75.9 years (SD 10.8) and 65.9% of patients (1,574/2,388) were female. The algorithms included patient and injury characteristics. Six algorithms were developed, internally validated and evaluated across discrimination (c-statistic; discriminative ability between those with risk of mortality and those without), calibration (observed outcome compared to the predicted probability), and the Brier score (composite of discrimination and calibration).ResultsThe developed algorithms distinguished between patients at high and low risk for 90-day and one-year mortality. The penalized logistic regression algorithm had the best performance metrics for both 90-day (c-statistic 0.80, calibration slope 0.95, calibration intercept -0.06, and Brier score 0.039) and one-year (c-statistic 0.76, calibration slope 0.86, calibration intercept -0.20, and Brier score 0.074) mortality prediction in the hold-out set.ConclusionUsing high-quality data, the ML-based prediction models accurately predicted 90-day and one-year mortality in patients aged 50 years or older with a FNF. The final models must be externally validated to assess generalizability to other populations, and prospectively evaluated in the process of shared decision-making.Cite this article: Bone Jt Open 2023;4(3):168–181.

show abstract

Intercontinental validation of a clinical prediction model for predicting 90-day and 2-year mortality in an Israeli cohort of 2033 patients with a femoral neck fracture aged 65 or above

Oosterhoff

Karhade

Groot

et al. 2023

Eur J Trauma Emerg Surg

Self Cite

View full text Add to dashboard Cite

Purpose Mortality prediction in elderly femoral neck fracture patients is valuable in treatment decision-making. A previously developed and internally validated clinical prediction model shows promise in identifying patients at risk of 90-day and 2-year mortality. Validation in an independent cohort is required to assess the generalizability; especially in geographically distinct regions. Therefore we questioned, is the SORG Orthopaedic Research Group (SORG) femoral neck fracture mortality algorithm externally valid in an Israeli cohort to predict 90-day and 2-year mortality? Methods We previously developed a prediction model in 2022 for estimating the risk of mortality in femoral neck fracture patients using a multicenter institutional cohort of 2,478 patients from the USA. The model included the following input variables that are available on clinical admission: age, male gender, creatinine level, absolute neutrophil, hemoglobin level, international normalized ratio (INR), congestive heart failure (CHF), displaced fracture, hemiplegia, chronic obstructive pulmonary disease (COPD), history of cerebrovascular accident (CVA) and beta-blocker use. To assess the generalizability, we used an intercontinental institutional cohort from the Sheba Medical Center in Israel (level I trauma center), queried between June 2008 and February 2022. Generalizability of the model was assessed using discrimination, calibration, Brier score, and decision curve analysis. Results The validation cohort included 2,033 patients, aged 65 years or above, that underwent femoral neck fracture surgery. Most patients were female 64.8% (n = 1317), the median age was 81 years (interquartile range = 75–86), and 80.4% (n = 1635) patients sustained a displaced fracture (Garden III/IV). The 90-day mortality was 9.4% (n = 190) and 2-year mortality was 30.0% (n = 610). Despite numerous baseline differences, the model performed acceptably to the validation cohort on discrimination (c-statistic 0.67 for 90-day, 0.67 for 2-year), calibration, Brier score, and decision curve analysis. Conclusions The previously developed SORG femoral neck fracture mortality algorithm demonstrated good performance in an independent intercontinental population. Current iteration should not be relied on for patient care, though suggesting potential utility in assessing patients at low risk for 90-day or 2-year mortality. Further studies should evaluate this tool in a prospective setting and evaluate its feasibility and efficacy in clinical practice. The algorithm can be freely accessed: https://sorg-apps.shinyapps.io/hipfracturemortality/. Level of evidence Level III, Prognostic study.

show abstract

Development and internal validation of a clinical prediction model using machine learning algorithms for 90 day and 2 year mortality in femoral neck fracture patients aged 65 years or above

Cited by 17 publications

References 58 publications

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction

Artificial Intelligence for Hip Fracture Detection and Outcome Prediction

Development of machine-learning algorithms for 90-day and one-year mortality prediction in the elderly with femoral neck fractures based on the HEALTH and FAITH trials

Intercontinental validation of a clinical prediction model for predicting 90-day and 2-year mortality in an Israeli cohort of 2033 patients with a femoral neck fracture aged 65 or above

Contact Info

Product

Resources

About