The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard

Agharia, Suzen; Szatkowski, Jan; Fraval, Andrew; Stevens, Jarrad; Zhou, Yushy

doi:10.1016/j.jor.2023.11.063

Journal of Orthopaedics

2024

DOI: 10.1016/j.jor.2023.11.063

|View full text |Cite

The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard

Suzen Agharia,

Jan Szatkowski,

Andrew Fraval

et al.

Help me understand this report

Search citation statements

Order By: Relevance

Paper Sections

Select...

Citation Types

Supporting

Mentioning

Contrasting

Year Published

2024

2025

Publication Types

Select...

Article5

Relationship

Self Cite0

Independent5

Authors

Journals

Cited by 8 publications

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

Evaluating the performance and clinical decision‐making impact of ChatGPT‐4 in reproductive medicine

Chen,

Zeng,

et al. 2024

Intl J Gynecology & Obste

View full text Add to dashboard Cite

BackgroundChatGPT, a sophisticated language model developed by OpenAI, has the potential to offer professional and patient‐friendly support. We aimed to assess the accuracy and reproducibility of ChatGPT‐4 in answering questions related to knowledge, management, and support within the field of reproductive medicine.MethodsChatGPT‐4 was used to respond to queries sourced from a domestic attending physician examination database, as well as to address both local and international treatment guidelines within the field of reproductive medicine. Each response generated by ChatGPT‐4 was independently evaluated by a trio of experts specializing in reproductive medicine. The experts used four qualitative measures—relevance, accuracy, completeness, and understandability—to assess each response.ResultsWe found that ChatGPT‐4 demonstrated extensive knowledge in reproductive medicine, with median scores for relevance, accuracy, completeness, and comprehensibility of objective questions being 4, 3.5, 3, and 3, respectively. However, the composite accuracy rate for multiple‐choice questions was 63.38%. Significant discrepancies were observed among the three experts' scores across all four measures. Expert 1 generally provided higher and more consistent scores, while Expert 3 awarded lower scores for accuracy. ChatGPT‐4's responses to both domestic and international guidelines showed varying levels of understanding, with a lack of knowledge on regional guideline variations. However, it offered practical and multifaceted advice regarding next steps and adjusting to new guidelines.ConclusionsWe analyzed the strengths and limitations of ChatGPT‐4's responses on the management of reproductive medicine and relevant support. ChatGPT‐4 might serve as a supplementary informational tool for patients and physicians to improve outcomes in the field of reproductive medicine.

show abstract

Evaluating the performance and clinical decision‐making impact of ChatGPT‐4 in reproductive medicine

Chen,

Zeng,

et al. 2024

Intl J Gynecology & Obste

View full text Add to dashboard Cite

show abstract

ChatGPT, Bard, and Bing Chat Are Large Language Processing Models That Answered Orthopaedic In-Training Examination Questions With Similar Accuracy to First-Year Orthopaedic Surgery Residents

Guerra,

Hofmann,

et al. 2024

Arthroscopy: The Journal of Arthroscopic & Related Surgery

View full text Add to dashboard Cite

Effectiveness of retrieval augmented generation-based large language models for generating construction safety information

Uhm,

Kim,

Ahn

et al. 2025

Automation in Construction

View full text Add to dashboard Cite

scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.

Contact Info

customersupport@researchsolutions.com

10624 S. Eastern Ave., Ste. A-614

Henderson, NV 89052, USA

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Blog Terms and Conditions API Terms Privacy Policy Contact Cookie Preferences Do Not Sell or Share My Personal Information

Made with 💙 for researchers

Part of the Research Solutions Family.

The ability of artificial intelligence tools to formulate orthopaedic clinical decisions in comparison to human clinicians: An analysis of ChatGPT 3.5, ChatGPT 4, and Bard

Cited by 8 publications

References 40 publications

Evaluating the performance and clinical decision‐making impact of ChatGPT‐4 in reproductive medicine

Evaluating the performance and clinical decision‐making impact of ChatGPT‐4 in reproductive medicine

ChatGPT, Bard, and Bing Chat Are Large Language Processing Models That Answered Orthopaedic In-Training Examination Questions With Similar Accuracy to First-Year Orthopaedic Surgery Residents

Effectiveness of retrieval augmented generation-based large language models for generating construction safety information

Contact Info

Product

Resources

About