Capacity for large language model chatbots to aid in orthopedic management, research, and patient queries

Abstract

Large language model (LLM) chatbots possess a remarkable capacity to synthesize complex information into concise, digestible summaries across a wide range of orthopedic subject matter. As LLM chatbots become widely available they will serve as a powerful, accessible resource that patients, clinicians, and researchers may reference to obtain information about orthopedic science and clinical management. Here, we examined the performance of three well‐known and easily accessible chatbots–ChatGPT, Bard, and Bing AI–in responding to inquiries relating to clinical management and orthopedic concepts. Although all three chatbots were found to be capable of generating relevant responses, ChatGPT outperformed Bard and BingAI in each category due to its ability to provide accurate and complete responses to orthopedic queries. Despite their promising applications in clinical management, shortcomings observed included incomplete responses, lack of context, and outdated information. Nonetheless, the ability for these LLM chatbots to address these inquires has largely yet to be evaluated and will be critical for understanding the risks and opportunities of LLM chatbots in orthopedics.

Comparative Analysis of Artificial Intelligence Virtual Assistant and Large Language Models in Post-Operative Care

“…Artificial intelligence (AI)-enabled tools have emerged as valuable assets in healthcare, particularly in patient management and individualized decision-making processes [7,8]. Numerous initiatives have focused on developing systems capable of engaging in meaningful, intelligent dialogues with patients [3,9]. Chatbots have become one of the most valuable information sources for patients worldwide, especially with recent technological advancements.…”

“…That said, there is a substantial body of research focused on the application of LLMs and AIVAs in specialized patient education tasks. These studies evaluate the feasibility, accuracy, and suitability of these technologies for responding to inquiries across various medical fields [9,15,[20][21][22][23]. Some studies have enhanced chatbots using LLMs, successfully simulating the patient-physician dynamic.…”