Radiology Gets Chatty: The ChatGPT Saga Unfolds

Grewal, Harpreet; Dhillon, Gagandeep; Monga, Varun; Sharma, Pranjal; Buddhavarapu, Venkata S; Sidhu, Gurmanpreet; Kashyap, Rahul

doi:10.7759/cureus.40135

Cited by 37 publications

(30 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Grewal et al [15] tested GPT-4 application in radiology across several fields, including patient education. GPT-4 generated patient-oriented explanations of radiological findings, and assisted in patient inquiries.…”

Section: Resultsmentioning

confidence: 99%

Large language models in simplifying radiological reports: systematic review

Artsi,

Sorin,

Konen

et al. 2024

Preprint

View full text Add to dashboard Cite

ObjectivesSimplifying medical information to make it understandable for patients, specifically in the case of radiology reports, is challenging. It requires time and effort from medical personnel. This systematic review focuses on the application of large language models (LLMs) in generating simplified radiological imaging reports, as well as answering patient inquiries regarding radiological procedures.Materials and MethodsThe authors searched for studies published up to January 2024. Search terms focused on LLMs generated simplified radiological reports and answers to patient inquiries regarding radiological procedures. MEDLINE was used as a search database.ResultsOverall, eight studies published between May 2023 and November 2023 were included. All studies showed that LLMs can produce simplified medical information for patients. Four studies (50%) used GPT-3.5, Two studies (25%) conducted a comparative analysis between GPT-3.5 and GPT-4. One study (12.5%) examined Microsoft Bing. One study (12.5%) utilized GPT-4. Four studies (50%) used LLMs to simplify radiological reports. Four studies (50%) used LLMs to answer patient questions regarding radiological procedures. Only two studies (25%) used patients to evaluate the LLMs output. One study (12.5%) compared their initial prompt with optimized prompt. Five studies (62.5%) showed missing, inaccurate and potentially harmful AI outputs.ConclusionLLMs can be used to simplify medical imaging reports and procedures, for improved patient comprehension. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed.

show abstract

Section: Resultsmentioning

confidence: 99%

Large language models in simplifying radiological reports: systematic review

Artsi,

Sorin,

Konen

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…Another contributing factor to inaccuracy is the outdated knowledge base used to train LLMs [21,25,30,41]. ChatGPT based on GPT3.5 was pre-trained by using data collected until 2021 and does not support Internet connection [49], making it unable to perform appropriately on questions regarding events that happened after 2021 [42].…”

Section: Resultsmentioning

confidence: 99%

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

Wang,

Wan,

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: The launch of the Chat Generative Pre-trained Transformer (ChatGPT) in November 2022 has attracted public attention and academic interest to large language models (LLMs), facilitating the emergence of many other innovative LLMs. These LLMs have been applied in various fields, including healthcare. Numerous studies have since been conducted regarding how to employ state-of-the-art LLMs in health-related scenarios to assist patients, doctors, and public health administrators. Objective: This review aims to summarize the applications and concerns of applying conversational LLMs in healthcare and provide an agenda for future research on LLMs in healthcare. Methods: We utilized PubMed, ACM, and IEEE digital libraries as primary sources for this review. We followed the guidance of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRIMSA) to screen and select peer-reviewed research articles that (1) were related to both healthcare applications and conversational LLMs and (2) were published before September 1st, 2023, the date when we started paper collection and screening. We investigated these papers and classified them according to their applications and concerns. Results: Our search initially identified 820 papers according to targeted keywords, out of which 65 papers met our criteria and were included in the review. The most popular conversational LLM was ChatGPT from OpenAI (60), followed by Bard from Google (1), Large Language Model Meta AI (LLaMA) from Meta (1), and other LLMs (5). These papers were classified into four categories in terms of their applications: 1) summarization, 2) medical knowledge inquiry, 3) prediction, and 4) administration, and four categories of concerns: 1) reliability, 2) bias, 3) privacy, and 4) public acceptability. There are 49 (75%) research papers using LLMs for summarization and/or medical knowledge inquiry, and 58 (89%) research papers expressing concerns about reliability and/or bias. We found that conversational LLMs exhibit promising results in summarization and providing medical knowledge to patients with a relatively high accuracy. However, conversational LLMs like ChatGPT are not able to provide reliable answers to complex health-related tasks that require specialized domain expertise. Additionally, no experiments in our reviewed papers have been conducted to thoughtfully examine how conversational LLMs lead to bias or privacy issues in healthcare research. Conclusions: Future studies should focus on improving the reliability of LLM applications in complex health-related tasks, as well as investigating the mechanisms of how LLM applications brought bias and privacy issues. Considering the vast accessibility of LLMs, legal, social, and technical efforts are all needed to address concerns about LLMs to promote, improve, and regularize the application of LLMs in healthcare.

show abstract

“…A n o t h e r u s e o f C h a t G P T c o u l d b e t o h e l p p h y s i c i a n s g e n e r a t e r e p o r t s , m a k e t h e m comprehendible for patients [34] or even write them in another language. This has been shown especially in the field of Radiology [35,36]. The sequencing of the MIR Examination questions is crucial in understanding GPT-4's performance.…”

Section: Discussionmentioning

confidence: 97%

Evaluating ChatGPT Efficacy in Navigating the Spanish Medical Residency Entrance Examination (MIR): A New Horizon for AI in Clinical Medicine

Guillen-Grima,

Guillen-Aguinaga,

Guillen-Aguinaga

et al. 2023

Preprint

View full text Add to dashboard Cite

The rapid progress in artificial intelligence, machine learning, and natural language processing has led to the emergence of increasingly sophisticated large language models (LLMs) enabling their use in healthcare. The study assesses the performance of two LLMs: the GPT-3.5 and GPT-4 models in passing the medical examination for access to medical specialist training in Spain MIR. Our objectives included gauging the model's overall performance, analyzing discrepancies across different medical specialties, discerning between theoretical and practical questions, estimating error proportions, and assessing the hypothetical severity of errors committed by a physician. We studied the 2022 Spanish MIR examination after excluding those questions requiring image evaluations or having acknowledged errors. The remaining 182 questions were presented to the LLM ChatGPT4 and GPT-3.5 in Spanish and English. Logistic regression models analyzed the relationships between question length and question sequence d performance. GPT-4 outperformed GPT -3.5, scoring 86.81% in Spanish (p<0.001). English translations had a slightly enhanced performance. Among medical specialties, GPT-4 achieved a 100% correct response rate in several areas, with specialties like Pharmacology, ICU, and Infectious Diseases showing lower performance. The error analysis revealed that while a 13.2% error rate existed, gravest categories like "error requiring intervention to sustain life" and "error resulting in death" had a 0% rate. Conclusions: GPT-4 performs robustly on the Spanish MIR examination, varying its capability to discriminate knoweldge across specialties. While the model's high success rate is commendable, understanding the error severity is critical, especially when considering AI's potential role in real-world medical practice and its implication on patient safety.

show abstract

Radiology Gets Chatty: The ChatGPT Saga Unfolds

Cited by 37 publications

References 16 publications

Large language models in simplifying radiological reports: systematic review

Large language models in simplifying radiological reports: systematic review

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

Evaluating ChatGPT Efficacy in Navigating the Spanish Medical Residency Entrance Examination (MIR): A New Horizon for AI in Clinical Medicine

Contact Info

Product

Resources

About