Exploring the Role of a Large Language Model on Carpal Tunnel Syndrome Management: An Observation Study of ChatGPT

Seth, Ishith; Xie, Yi; Rodwell, Aaron; Gracias, Dylan; Bulloch, Gabriella; Hunter-Smith, David J.; Rozen, Warren M.

doi:10.1016/j.jhsa.2023.07.003

Cited by 32 publications

(25 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…An overview of the presence of codes for each study is provided in Supplementary Section 3. The majority of articles investigated the use and feasibility of LLMs as medical chatbots (n=84/89, 94.4%) 13,[64][65][66]68,69,[98][99][100][101][102][103][104][105][106][107][108][109][110][111] , while fewer reports additionally or exclusively focused on the generation of patient information (n=19/89, 21.4%) 24,31,43,48,49,57,59,62,67,70,79,[88][89][90][91]97,102,106,107 , including clinical documentation such as informed consent forms (n=5/89, 5.6%) 43,67,91,97,102 and discharge instructions (n=1/89, 1.1%) 31 , or translation/summarization tasks of medical texts (n=5/89, 5.6%) 24,49,…”

Section: Applications Of Large Language Modelsmentioning

confidence: 99%

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges

Busch,

Hoffmann,

Rueger

et al. 2024

Preprint

View full text Add to dashboard Cite

The introduction of large language models (LLMs) into clinical practice promises to improve patient education and empowerment, thereby personalizing medical care and broadening access to medical knowledge. Despite the popularity of LLMs, there is a significant gap in systematized information on their use in patient care. Therefore, this systematic review aims to synthesize current applications and limitations of LLMs in patient care using a data-driven convergent synthesis approach. We searched 5 databases for qualitative, quantitative, and mixed methods articles on LLMs in patient care published between 2022 and 2023. From 4,349 initial records, 89 studies across 29 medical specialties were included, primarily examining models based on the GPT-3.5 (53.2%, n=66 of 124 different LLMs examined per study) and GPT-4 (26.6%, n=33/124) architectures in medical question answering, followed by patient information generation, including medical text summarization or translation, and clinical documentation. Our analysis delineates two primary domains of LLM limitations: design and output. Design limitations included 6 second-order and 12 third-order codes, such as lack of medical domain optimization, data transparency, and accessibility issues, while output limitations included 9 second-order and 32 third-order codes, for example, non-reproducibility, non-comprehensiveness, incorrectness, unsafety, and bias. In conclusion, this study is the first review to systematically map LLM applications and limitations in patient care, providing a foundational framework and taxonomy for their implementation and evaluation in healthcare settings.

show abstract

Section: Applications Of Large Language Modelsmentioning

confidence: 99%

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges

Busch,

Hoffmann,

Rueger

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Articles published in peerreviewed journals that overly exaggerate the true capabilities of ChatGPT might serve as evidence of this [7,8]. Despite such claims, ChatGPT may not be effective in conducting a literature review, selecting relevant articles, diagnosing clinical conditions or finding proper references [7,9]. Similarly, a previously published editorial [1] focused on highlighting its limitations and the associated risks of its irresponsible use.…”

Section: The Current Stance Of the Scientific Community On Chatgptmentioning

confidence: 99%

“…It cannot create original or novel content or critically review a written text other than for formatting or outlining purposes, nor can it distinguish right from wrong. It can misinterpret previously available data and make up stories, individuals or false scientific references, especially when asked for tasks it was not originally intended for [9,22]. It can create titles for manuscripts that sound close to real, being complete with real authors, institutions and real journals.…”

Section: Chatgpt Model and Its Benefits In Academic Writingmentioning

confidence: 99%

“…It can create titles for manuscripts that sound close to real, being complete with real authors, institutions and real journals. As dangerous as this may sound, it is relatively easy to identify if the user is accustomed to its limitations and has a good sense of what it can be used for, considering its abilities [9,22]. The findings of a prior study suggested the need for caution when obtaining answers related to orthopaedic surgery from ChatGPT.…”

Section: Chatgpt Model and Its Benefits In Academic Writingmentioning

confidence: 99%

“…It cannot replace human expertise, assessment or hypothesis-driven scientific inquiry. The current version of ChatGPT will provide the user with fake references if the user specifically demands citations [9,22]. It cannot provide plausible references to its statements, as it is a language-processing tool and not a researcher.…”

Section: Accountability Plagiarism Risks Of Chatgpt and Culture Of Hy...mentioning

confidence: 99%

See 2 more Smart Citations

Embrace responsible ChatGPT usage to overcome language barriers in academic writing

Kayaalp,

Ollivier,

Winkler

et al. 2023

Knee surg. sports traumatol. arthrosc.

View full text Add to dashboard Cite

show abstract

Assessment of Artificial Intelligence Chatbot Responses to Common Patient Questions on Bone Sarcoma

Khabaz,

Newman‐Hung,

Kallini

et al. 2024

Journal of Surgical Oncology

View full text Add to dashboard Cite

Background and ObjectivesThe potential impacts of artificial intelligence (AI) chatbots on care for patients with bone sarcoma is poorly understood. Elucidating potential risks and benefits would allow surgeons to define appropriate roles for these tools in clinical care.MethodsEleven questions on bone sarcoma diagnosis, treatment, and recovery were inputted into three AI chatbots. Answers were assessed on a 5‐point Likert scale for five clinical accuracy metrics: relevance to the question, balance and lack of bias, basis on established data, factual accuracy, and completeness in scope. Responses were quantitatively assessed for empathy and readability. The Patient Education Materials Assessment Tool (PEMAT) was assessed for understandability and actionability.ResultsChatbots scored highly on relevance (4.24) and balance/lack of bias (4.09) but lower on basing responses on established data (3.77), completeness (3.68), and factual accuracy (3.66). Responses generally scored well on understandability (84.30%), while actionability scores were low for questions on treatment (64.58%) and recovery (60.64%). GPT‐4 exhibited the highest empathy (4.12). Readability scores averaged between 10.28 for diagnosis questions to 11.65 for recovery questions.ConclusionsWhile AI chatbots are promising tools, current limitations in factual accuracy and completeness, as well as concerns of inaccessibility to populations with lower health literacy, may significantly limit their clinical utility.

show abstract

Exploring the Role of a Large Language Model on Carpal Tunnel Syndrome Management: An Observation Study of ChatGPT

Cited by 32 publications

References 6 publications

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges

Systematic Review of Large Language Models for Patient Care: Current Applications and Challenges

Embrace responsible ChatGPT usage to overcome language barriers in academic writing

Assessment of Artificial Intelligence Chatbot Responses to Common Patient Questions on Bone Sarcoma

Contact Info

Product

Resources

About