ChatGPT performance in laryngology and head and neck surgery: a clinical case-series

Lechien, Jerome R.; Georgescu, Bianca M.; Hans, Stephane; Chiesa-Estomba, Carlos M.

doi:10.1007/s00405-023-08282-5

Cited by 26 publications

(15 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Each clinical case included 4 basic elements: the medical history; the clinical examination; the technical diagnostic findings (from additional examinations); and the treatment, 4,16 all of which involve the psychosocial context of the patient. The findings of these features lead to a variation in the degree of complexity, as the complexity may increase when the case contains distracting information.…”

Section: Level Of Complexity Of the Casementioning

confidence: 99%

“…Many scoring systems were developed to rate the degree of complexity of real clinical cases. In this study, the raters used a modified version of the General Items off the Amsterdam Clinical Challenge Scale (ACCS) test 4,16 to rate the complexity of clinical cases submitted to ChatGPT-4. The baseline version of ACCS included 6 generic items: previous history/actual context; problem presented; communication (patient complaints and responses); physical examination (typical vs atypical signs); patient management (adequate vs complicated management); and prevention.…”

Section: Level Of Complexity Of the Casementioning

confidence: 99%

“…ChatGPT has been used to respond to both simple and complicated questions related to medical examinations, and clinical vignettes, and may improve scientific reports through spelling correction or referencing. [1][2][3] The accessibility and popularity of ChatGPT may encourage patients to use them for education prior to a medical consultation, 4 and young practitioners may consider ChatGPT-4 as an adjunctive clinical tool for improving their practice. To date, only few small cohort studies investigated the consistency and performance of ChatGPT in the management of common otolaryngological conditions.…”

mentioning

confidence: 99%

“…To date, only few small cohort studies investigated the consistency and performance of ChatGPT in the management of common otolaryngological conditions. 1,2,4 In this study, we investigated the performance of ChatGPT-4 in the diagnostic and therapeutic management of true clinical cases and, consequently, its consistency in providing medical information.…”

mentioning

confidence: 99%

See 3 more Smart Citations

Performance and Consistency of ChatGPT‐4 Versus Otolaryngologists: A Clinical Case Series

Lechien,

Naunheim,

Maniaci

et al. 2024

Otolaryngol.--head neck surg.

View full text Add to dashboard Cite

ObjectiveTo study the performance of Chatbot Generative Pretrained Transformer‐4 (ChatGPT‐4) in the management of cases in otolaryngology–head and neck surgery.Study DesignProspective case series.SettingMulticenter University Hospitals.MethodsHistory, clinical, physical, and additional examinations of adult outpatients consulting in otolaryngology departments of CHU Saint‐Pierre and Dour Medical Center were presented to ChatGPT‐4, which was interrogated for differential diagnoses, management, and treatment(s). According to specialty, the ChatGPT‐4 responses were assessed by 2 distinct, blinded board‐certified otolaryngologists with the Artificial Intelligence Performance Instrument.ResultsOne hundred cases were presented to ChatGPT‐4. ChaGPT‐4 indicated a mean of 3.34 (95% confidence interval [CI]: 3.09, 3.59) additional examinations per patient versus 2.10 (95% CI: 1.76, 2.34; P = .001) for the practitioners. There was strong consistency (k > 0.600) between otolaryngologists and ChatGPT‐4 for the indication of upper aerodigestive tract endoscopy, positron emission tomography and computed tomography, audiometry, tympanometry, and psychophysical evaluations. Primary diagnosis was correctly performed by ChatGPT‐4 in 38% to 86% of cases depending on subspecialty. Additional examinations indicated by ChatGPT‐4 were pertinent and necessary in 8% to 31% of cases, while the treatment regimen was pertinent in 12% to 44% of cases. The performance of ChatGPT‐4 was not influenced by the human‐reported level of difficulty of clinical cases.ConclusionChatGPT‐4 may be a promising adjunctive tool in otolaryngology, providing extensive documentation about additional examinations, primary and differential diagnoses, and treatments. The ChatGPT‐4 is more effective in providing a primary diagnosis, and less effective in the selection of additional examinations and treatments.

show abstract

Section: Level Of Complexity Of the Casementioning

confidence: 99%

Section: Level Of Complexity Of the Casementioning

confidence: 99%

mentioning

confidence: 99%

mentioning

confidence: 99%

See 2 more Smart Citations

Performance and Consistency of ChatGPT‐4 Versus Otolaryngologists: A Clinical Case Series

Lechien,

Naunheim,

Maniaci

et al. 2024

Otolaryngol.--head neck surg.

View full text Add to dashboard Cite

show abstract

“…There is a growing body of literature showing the impressive ability of ChatGPT in answering clinical questions on a wide range of topics including cirrhosis, bariatric surgery, laryngology and orthopaedic surgery. (15)(16)(17)(18) There is little to no literature examining the performance of ChatGPT or other LLMs regarding nutrition-related questions. Examining the performance of LLMs, like ChatGPT, in the realm of nutrition will be critical to understanding the future role and limitations of this technology.…”

Section: Introductionmentioning

confidence: 99%

Examining the Accuracy and Reproducibility of Responses to Nutrition Questions Related to Inflammatory Bowel Disease by Generative Pre-trained Transformer-4 (GPT-4)

Samaan,

Issokson,

Feldman

et al. 2023

Preprint

View full text Add to dashboard Cite

Background and AimsGenerative Pre-trained Transformer-4 (GPT-4) is a large language model (LLM) trained on a variety of topics, including the medical literature. Nutrition plays a critical role in managing inflammatory bowel disease (IBD), with an unmet need for nutrition-related patient education resources. The aim of this study is to examine the accuracy and reproducibility of responses by GPT-4 to patient nutrition questions related to IBD.MethodsQuestions were curated from adult IBD clinic visits, Facebook, and Reddit. Two IBD-focused registered dieticians independently graded the accuracy and reproducibility of GPT-4’s responses while a third senior IBD-focused registered dietitian arbitrated. To ascertain reproducibility, each question was inputted twice into the model. Descriptive analysis is presented as counts and proportions.ResultsIn total, 88 questions were included. The model provided correct responses to 73/88 questions (83.0%), with 61 (69.0%) graded as comprehensive. A total of 15/88 (17%) responses were graded as mixed with correct and incorrect/outdated data. When examined by category, the model provided comprehensive responses to 10 (62.5%) questions related to “Nutrition and diet needs for surgery”, 12 (92.3%) “Tube feeding and parenteral nutrition”, 11 (64.7%) “General diet questions”, 10 (50%) “Diet for reducing symptoms/inflammation” and 18 (81.8%) “Micronutrients/supplementation needs”. The model provided reproducible responses to 81/88 (92.0%) questions.ConclusionGPT-4 provided comprehensive responses to the majority of questions, demonstrating the promising potential of LLMs as supplementary tools for IBD patients seeking nutrition-related information. However, 17% of responses contained incorrect information, highlighting the need for continuous refinement and validation of LLMs prior to incorporation into clinical practice. Future studies should focus on leveraging LLMs to enhance patient outcomes. Furthermore, efforts promoting patient and healthcare professional proficiency in using LLMs are essential to maximize their efficacy and facilitate personalized care.

show abstract

ChatGPT‐4 performance in rhinology: A clinical case series

Radulesco,

Saibene,

Michel

et al. 2024

Int Forum Allergy Rhinol

Self Cite

View full text Add to dashboard Cite

Keypoints Chatbot Generative Pre‐trained Transformer (ChatGPT)‐4 indicated more than twice additional examinations than practitioners in the management of clinical cases in rhinology. The consistency between ChatGPT‐4 and practitioner in the indication of additional examinations may significantly vary from one examination to another. The ChatGPT‐4 proposed a plausible and correct primary diagnosis in 62.5% cases, while pertinent and necessary additional examinations and therapeutic regimen were indicated in 7.5%–30.0% and 7.5%–32.5% of cases, respectively. The stability of ChatGPT‐4 responses is moderate‐to‐high. The performance of ChatGPT‐4 was not influenced by the human‐reported level of difficulty of clinical cases.

show abstract

ChatGPT performance in laryngology and head and neck surgery: a clinical case-series

Cited by 26 publications

References 18 publications

Performance and Consistency of ChatGPT‐4 Versus Otolaryngologists: A Clinical Case Series

Performance and Consistency of ChatGPT‐4 Versus Otolaryngologists: A Clinical Case Series

Examining the Accuracy and Reproducibility of Responses to Nutrition Questions Related to Inflammatory Bowel Disease by Generative Pre-trained Transformer-4 (GPT-4)

ChatGPT‐4 performance in rhinology: A clinical case series

Contact Info

Product

Resources

About