The quality and readability of patient information provided by ChatGPT: can AI reliably explain common ENT operations?

Abou-Abdallah, Michel; Dar, Talib; Mahmudzade, Yasamin; Michaels, Joshua; Talwar, Rishi; Tornari, Chrysostomos

doi:10.1007/s00405-024-08598-w

Cited by 6 publications

(1 citation statement)

References 21 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our findings confirmed that LLMs hold promise in trauma triage, but improvements are necessary for reliable implementation in clinical practice. During the last year, the application of LLMs in healthcare has dramatically increased [19][20][21][22]. The deployment of these models raises some concerns regarding potential biases in algorithmic decision making, adherence to medical regulatory frameworks, risk of hallucination, and privacy issues [23][24][25][26].…”

Section: Discussionmentioning

confidence: 99%

The Role of Large Language Models (LLMs) in Providing Triage for Maxillofacial Trauma Cases: A Preliminary Study

Frosolini,

Catarzi,

Benedetti

et al. 2024

Diagnostics

View full text Add to dashboard Cite

Background: In the evolving field of maxillofacial surgery, integrating advanced technologies like Large Language Models (LLMs) into medical practices, especially for trauma triage, presents a promising yet largely unexplored potential. This study aimed to evaluate the feasibility of using LLMs for triaging complex maxillofacial trauma cases by comparing their performance against the expertise of a tertiary referral center. Methods: Utilizing a comprehensive review of patient records in a tertiary referral center over a year-long period, standardized prompts detailing patient demographics, injury characteristics, and medical histories were created. These prompts were used to assess the triage suggestions of ChatGPT 4.0 and Google GEMINI against the center’s recommendations, supplemented by evaluating the AI’s performance using the QAMAI and AIPI questionnaires. Results: The results in 10 cases of major maxillofacial trauma indicated moderate agreement rates between LLM recommendations and the referral center, with some variances in the suggestion of appropriate examinations (70% ChatGPT and 50% GEMINI) and treatment plans (60% ChatGPT and 45% GEMINI). Notably, the study found no statistically significant differences in several areas of the questionnaires, except in the diagnosis accuracy (GEMINI: 3.30, ChatGPT: 2.30; p = 0.032) and relevance of the recommendations (GEMINI: 2.90, ChatGPT: 3.50; p = 0.021). A Spearman correlation analysis highlighted significant correlations within the two questionnaires, specifically between the QAMAI total score and AIPI treatment scores (rho = 0.767, p = 0.010). Conclusions: This exploratory investigation underscores the potential of LLMs in enhancing clinical decision making for maxillofacial trauma cases, indicating a need for further research to refine their application in healthcare settings.

show abstract