GPT-4 as a Source of Patient Information for Anterior Cervical Discectomy and Fusion: A Comparative Analysis Against Google Web Search

Mastrokostas, Paul G.; Mastrokostas, Leonidas E.; Emara, Ahmed K.; Wellington, Ian J.; Ginalis, Elizabeth; Houten, John K.; Khalsa, Amrit S.; Saleh, Ahmed; Razi, Afshin E.; Ng, Mitchell K.

doi:10.1177/21925682241241241

Cited by 4 publications

(1 citation statement)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In readability test, the readability of the rehabilitation programs generated by GPT-4 was deemed fairly difficult, consistent with findings from earlier studies 45 . Compared to a survey in another study 46 , the FRES for results generated by GPT-4 in this study was lower than those from Google, while the FKGL was higher, suggesting that without specific instructions, the readability of treatment plans produced by GPT-4 is inferior to Google. Studies have shown that GPT-4 can simplify health information and make it more readable 47 .…”

Section: Discussioncontrasting

confidence: 89%

Quality and Comprehensibility of Large Language Model Responses to Common Patient Questions Regarding Musculoskeletal Disorders (Preprint)

Fu,

Chen,

You

et al. 2024

Preprint

View full text Add to dashboard Cite

BACKGROUND Artificial intelligence (AI) and large language models (LLMs) are emerging as the transformative force in various fields, notably in medicine. Their effectiveness in creating physical exercise rehabilitation program and providing information on musculoskeletal (MSK) disorders has yet to be fully explored. OBJECTIVE To assess the quality and readability of an LLM’s responses to consultation questions addressing the various phases throughout the entire clinical process experienced by patients with chronic musculoskeletal disorders. METHODS This cross-sectional study retrieved frequently asked questions from Google (accessed September 3 to October 24, 2023) and randomly selected 25 adult patients suffering from chronic musculoskeletal pain. Three different clinical scenario questions were designed to simulate the entire process of real-world clinical consultations. These questions were used as queries for an AI LLM, ChatGPT version 4.0 (accessed September 23 to December 24, 2023), to prompt LLM-generate responses. The quality of the responses was evaluated by two independent orthopedic clinicians with DISCERN instrument, and the readability was assessed on the WedFX tool website (accessed December 14 to December 20, 2023). Statistical analysis was conducted from January to April 2024. RESULTS Of the 98 generated programs, the response format was relatively fixed, based on the queries provided to the LLM. The mean (SD) DISCERN scores assigned by the two doctors were 52.49 (8.57) and 51.50 (8.64), respectively, with all scores ranging from 32 to 67. The analysis of variance between the two physicians (p=0.42) and among the five musculoskeletal disorders (p=0.08) showed no significant difference. The Cohen κ coefficient was calculated to be 0.73 (95% CI −0.710 to 0.756), signifying good internal agreement, while Cronbach's α value is 0.834, indicating good reliability. The mean (SD) scores across the six readability tools for all programs were as follows: FRES: 53.04 (16.23), FKGL: 10.15 (3.42), GF: 12.52 (3.41), SMOG: 9.87 (2.75), CLI: 13.28 (1.97), ARI: 10.48 (3.80). All these tools’ readability score were significantly different (worse) than the recommended reading level (p<0.05). CONCLUSIONS In this cross-sectional study, the LLM generated high quality physical exercise prescriptions throughout the entire consultation process of patients with musculoskeletal disorders. However, the lack of supporting materials and the readability of the responses was no sufficiently public-friendly. These findings suggest that, with professional physician evaluation and some improvements, LLMs could be potentially and widely applied in the field of orthopedics in the future.

show abstract

Section: Discussioncontrasting

confidence: 89%

Quality and Comprehensibility of Large Language Model Responses to Common Patient Questions Regarding Musculoskeletal Disorders (Preprint)

Fu,

Chen,

You

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

Large language models in neurosurgery: a systematic review and meta-analysis

Patil,

Serrato,

Chisvo

et al. 2024

Acta Neurochir

View full text Add to dashboard Cite

Large language models in patient education: a scoping review of applications in medicine

Aydin,

Karabacak,

Vlachos

et al. 2024

Front. Med.

View full text Add to dashboard Cite

IntroductionLarge Language Models (LLMs) are sophisticated algorithms that analyze and generate vast amounts of textual data, mimicking human communication. Notable LLMs include GPT-4o by Open AI, Claude 3.5 Sonnet by Anthropic, and Gemini by Google. This scoping review aims to synthesize the current applications and potential uses of LLMs in patient education and engagement.Materials and methodsFollowing the PRISMA-ScR checklist and methodologies by Arksey, O’Malley, and Levac, we conducted a scoping review. We searched PubMed in June 2024, using keywords and MeSH terms related to LLMs and patient education. Two authors conducted the initial screening, and discrepancies were resolved by consensus. We employed thematic analysis to address our primary research question.ResultsThe review identified 201 studies, predominantly from the United States (58.2%). Six themes emerged: generating patient education materials, interpreting medical information, providing lifestyle recommendations, supporting customized medication use, offering perioperative care instructions, and optimizing doctor-patient interaction. LLMs were found to provide accurate responses to patient queries, enhance existing educational materials, and translate medical information into patient-friendly language. However, challenges such as readability, accuracy, and potential biases were noted.DiscussionLLMs demonstrate significant potential in patient education and engagement by creating accessible educational materials, interpreting complex medical information, and enhancing communication between patients and healthcare providers. Nonetheless, issues related to the accuracy and readability of LLM-generated content, as well as ethical concerns, require further research and development. Future studies should focus on improving LLMs and ensuring content reliability while addressing ethical considerations.

show abstract

GPT-4 as a Source of Patient Information for Anterior Cervical Discectomy and Fusion: A Comparative Analysis Against Google Web Search

Cited by 4 publications

References 37 publications

Quality and Comprehensibility of Large Language Model Responses to Common Patient Questions Regarding Musculoskeletal Disorders (Preprint)

Quality and Comprehensibility of Large Language Model Responses to Common Patient Questions Regarding Musculoskeletal Disorders (Preprint)

Large language models in neurosurgery: a systematic review and meta-analysis

Large language models in patient education: a scoping review of applications in medicine

Contact Info

Product

Resources

About