Artificial Intelligence to Diagnose Tibial Plateau Fractures: An Intelligent Assistant for Orthopedic Physicians

Liu, Pengran; Zhang, Jiayao; Xue, Mingdi; Duan, Yu-Yu; Hu, Jialang; Liu, Songxiang; Xie, Yi; Wang, Honglin; Wang, Junwen; Huo, Tongtong; Ye, Zhewei

doi:10.1007/s11596-021-2501-4

Cited by 22 publications

(13 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Others have indicated opportunities for AI to leverage big data to obtain insights and develop strategies for managing specific diseases, including opioid use disorders [1]. Another example of recognition and interpretation was offered by Liu et al [10], in which AI and orthopaedic surgeons correctly identified a similar number of tibial plateau fractures (accuracy 0.91 versus 0.92). These use cases could improve efficiency and accuracy in diagnosis and treatment, ultimately leading to better patient outcomes.…”

Section: Discussionmentioning

confidence: 99%

Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT

Lum

2023

Clin Orthop Relat Res

View full text Add to dashboard Cite

Background Advances in neural networks, deep learning, and artificial intelligence (AI) have progressed recently. Previous deep learning AI has been structured around domain-specific areas that are trained on dataset-specific areas of interest that yield high accuracy and precision. A new AI model using large language models (LLM) and nonspecific domain areas, ChatGPT (OpenAI), has gained attention. Although AI has demonstrated proficiency in managing vast amounts of data, implementation of that knowledge remains a challenge. Questions/purposes (1) What percentage of Orthopaedic In-Training Examination questions can a generative, pretrained transformer chatbot (ChatGPT) answer correctly? (2) How does that percentage compare with results achieved by orthopaedic residents of different levels, and if scoring lower than the 10th percentile relative to 5th-year residents is likely to correspond to a failing American Board of Orthopaedic Surgery score, is this LLM likely to pass the orthopaedic surgery written boards? (3) Does increasing question taxonomy affect the LLM’s ability to select the correct answer choices? Methods This study randomly selected 400 of 3840 publicly available questions based on the Orthopaedic In-Training Examination and compared the mean score with that of residents who took the test over a 5-year period. Questions with figures, diagrams, or charts were excluded, including five questions the LLM could not provide an answer for, resulting in 207 questions administered with raw score recorded. The LLM’s answer results were compared with the Orthopaedic In-Training Examination ranking of orthopaedic surgery residents. Based on the findings of an earlier study, a pass-fail cutoff was set at the 10th percentile. Questions answered were then categorized based on the Buckwalter taxonomy of recall, which deals with increasingly complex levels of interpretation and application of knowledge; comparison was made of the LLM’s performance across taxonomic levels and was analyzed using a chi-square test. Results ChatGPT selected the correct answer 47% (97 of 207) of the time, and 53% (110 of 207) of the time it answered incorrectly. Based on prior Orthopaedic In-Training Examination testing, the LLM scored in the 40th percentile for postgraduate year (PGY) 1s, the eighth percentile for PGY2s, and the first percentile for PGY3s, PGY4s, and PGY5s; based on the latter finding (and using a predefined cutoff of the 10th percentile of PGY5s as the threshold for a passing score), it seems unlikely that the LLM would pass the written board examination. The LLM’s performance decreased as question taxonomy level increased (it answered 54% [54 of 101] of Tax 1 questions correctly, 51% [18 of 35] of Tax 2 questions correctly, and 34% [24 of 71] of Tax 3 questions correctly; p = 0.034). Conclusion Although this general-domain LLM has a low likelihood of passing the orthopaedic surgery board examination, testing performance and knowledge are comparable to that of a first-year orthopaedic surgery resident. The LLM's ability to provide accurate answers declines with increasing question taxonomy and complexity, indicating a deficiency in implementing knowledge. Clinical Relevance Current AI appears to perform better at knowledge and interpretation-based inquires, and based on this study and other areas of opportunity, it may become an additional tool for orthopaedic learning and education.

show abstract

Section: Discussionmentioning

confidence: 99%

Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT

Lum

2023

Clin Orthop Relat Res

View full text Add to dashboard Cite

show abstract

“…There have been a number of studies with different products on fracture detection using AI on plain radiographs [ 28 , 29 , 30 , 31 , 32 , 33 , 34 ]. Fractures are the leading type of missed diagnosis [ 35 ].…”

Section: Methodsmentioning

confidence: 99%

“…There was, however, a significant difference in the time spent assessing the X-rays, with the AI being 16-times faster than orthopedic physicians. The authors state that the working conditions for orthopedic physicians were not ideal to simulate real working conditions, since the study was conducted in nonemergency and under time-free constraints on the diagnosis process [ 29 ].…”

Section: Methodsmentioning

confidence: 99%

Artificial Intelligence in Orthopedic Radiography Analysis: A Narrative Review

et al. 2022

View full text Add to dashboard Cite

Artificial intelligence (AI) in medicine is a rapidly growing field. In orthopedics, the clinical implementations of AI have not yet reached their full potential. Deep learning algorithms have shown promising results in computed radiographs for fracture detection, classification of OA, bone age, as well as automated measurements of the lower extremities. Studies investigating the performance of AI compared to trained human readers often show equal or better results, although human validation is indispensable at the current standards. The objective of this narrative review is to give an overview of AI in medicine and summarize the current applications of AI in orthopedic radiography imaging. Due to the different AI software and study design, it is difficult to find a clear structure in this field. To produce more homogeneous studies, open-source access to AI software codes and a consensus on study design should be aimed for.

show abstract

“…Liu suggests that their AI algorithm would perform even better in clinical settings. Humans have been shown to be prone to missed diagnoses when under pressure or overworked, making AI a potentially useful tool in these scenarios[ 22 ].…”

Section: Image-based Ai Applicationmentioning

confidence: 99%

Application of artificial intelligence in trauma orthopedics: Limitation and prospects

Salimi,

Parry,

Shahrokhi

et al. 2023

World J Clin Cases

View full text Add to dashboard Cite

The varieties and capabilities of artificial intelligence and machine learning in orthopedic surgery are extensively expanding. One promising method is neural networks, emphasizing big data and computer-based learning systems to develop a statistical fracture-detecting model. It derives patterns and rules from outstanding amounts of data to analyze the probabilities of different outcomes using new sets of similar data. The sensitivity and specificity of machine learning in detecting fractures vary from previous studies. AI may be most promising in the diagnosis of less-obvious fractures that are more commonly missed. Future studies are necessary to develop more accurate and effective detection models that can be used clinically.

show abstract

Artificial Intelligence to Diagnose Tibial Plateau Fractures: An Intelligent Assistant for Orthopedic Physicians

Cited by 22 publications

References 26 publications

Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT

Can Artificial Intelligence Pass the American Board of Orthopaedic Surgery Examination? Orthopaedic Residents Versus ChatGPT

Artificial Intelligence in Orthopedic Radiography Analysis: A Narrative Review

Application of artificial intelligence in trauma orthopedics: Limitation and prospects

Contact Info

Product

Resources

About