The ensemble artificial intelligence (AI) method: Detection of hip fractures in AP pelvis plain radiographs by majority voting using a multi-center dataset

Abstract

Introduction This article was undertaken to explore the potential of AI in enhancing the diagnostic accuracy and efficiency in identifying hip fractures using X-ray radiographs. In the study, we trained three distinct deep learning models, and we utilized majority voting to evaluate their outcomes, aiming to yield the most reliable and precise diagnoses of hip fractures from X-ray radiographs. Methods An initial study was conducted of 10,849 AP pelvis X-rays obtained from five hospitals affiliated with Başkent University. Two expert orthopedic surgeons initially labeled 2,291 radiographs as fractures and 8,558 as non-fractures. The algorithm was trained on 6,943 (64%) radiographs, validated on 1,736 (16%) radiographs, and tested on 2,170 (20%) radiographs, ensuring an even distribution of fracture presence, age, and gender. We employed three advanced deep learning architectures, Xception (Model A), EfficientNet (Model B), and NfNet (Model C), with a final decision aggregated through a majority voting technique (Model D). Results For each model, we achieved the following metrics: For Model A: F1 Score 0.895, Accuracy 0.956, Specificity 0.973, Sensitivity 0.893. For Model B: F1 Score 0.900, Accuracy 0.960, Specificity 0.991, Sensitivity 0.845. For Model C: F1 Score 0.919, Accuracy 0.966, Specificity 0.984, Sensitivity 0.899. For Model D: F1 Score 0.929, Accuracy 0.971, Specificity 0.991, Sensitivity 0.897. We concluded that Model D (majority voting) achieved the best results in terms of the F1 score, accuracy, and specificity values. Conclusions Our study demonstrates that the results obtained by aggregating the decisions of multiple models through voting, rather than relying solely on the decision of a single algorithm, are more consistent. The practical application of these algorithms will be difficult due to ethical, legal, and confidentiality issues, despite the theoretical success achieved. Developing successful algorithms and methodologies should not be viewed as the ultimate goal; it is important to understand how these algorithms will be used in real-life situations. In order to achieve more consistent results, feedback from clinical practice will be helpful.

Comparison of artificial intelligence algorithm for the diagnosis of hip fracture on plain radiography with decision-making physicians: a validation study

“…We developed an AI system for hip fracture diagnosis that provides high accuracy, sensitivity, and specificity. 9 According to the F1 scores, the majority voting results were the most successful compared to the specialist groups’ averages and each AI model’s average. For the first time, the capacity of emergency medicine physicians and radiologists to diagnose hip fractures correctly was revealed.…”

“…In a previous study, we developed a decision support system that automatically detects hip fractures on x-ray images of the pelvis AP. 9 A large multicenter dataset of 19,583 radiographs collected from 5 centres was created to achieve success at the clinical expert level. The radiographs of patients < 16 years, those with implants, and those with lateral hip view were excluded.…”