The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models

Minssen, Timo; Vayena, Effy; Cohen, I. Glenn

doi:10.1001/jama.2023.9651

Cited by 90 publications

(35 citation statements)

References 5 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many areas of regulation lie outside the scope of this article, although there are several regulatory issues that are particularly important with respect to neurology and more broadly, medicine, that pertain to the technical and ethical challenges we raise. 19,47,48 Software as a medical device (SaMD) is defined by the International Medical Device Regulators Forum (IMDRF), as software that is not embedded within hardware and which performs medical tasks. Therefore, many medical LLMs would fall under this umbrella.…”

Section: Role Of Regulationmentioning

confidence: 99%

See 1 more Smart Citation

Large Language Models in Neurology Research and Future Practice

Romano,

Shih,

Paschalidis

et al. 2023

Neurology

View full text Add to dashboard Cite

Recent advancements in generative artificial intelligence, particularly using large language models (LLMs), are gaining increased public attention. We provide a perspective on the potential of LLMs to analyze enormous amounts of data from medical records and gain insights on specific topics in neurology. Additionally, we explore use cases for LLMs such as early diagnosis, supporting patient and caregivers, and acting as an assistant for clinicians. We point to the potential ethical and technical challenges raised by LLMs, such as concerns about privacy and data security, potential biases in the data for model training, and the need for careful validation of results. Researchers must consider these challenges and take steps to address them to ensure that their work is conducted in a safe and responsible manner. Despite these challenges, LLMs offer promising opportunities for improving care and treatment of various neurological disorders.

show abstract

Section: Role Of Regulationmentioning

confidence: 99%

Section: Technical and Ethical Challengesmentioning

confidence: 99%

Large Language Models in Neurology Research and Future Practice

Romano,

Shih,

Paschalidis

et al. 2023

Neurology

View full text Add to dashboard Cite

show abstract

“…We used images from peer-reviewed nuclear medicine journals, avoiding in-house patient data due to data privacy concerns with ChatGPT. 4 Yet, none of these images contained a color scale, and only 1 image indicated the patient orientation, leading to interpretation FIGURE 2. Examples of attempts to mark image findings.…”

Section: Discussionmentioning

confidence: 99%

“…3 Although ChatGPT is not designed for medical advice, unresolved concerns about data privacy, particularly for medical images, and legal liability remain. 4 Reports that ChatGPT is already being used by physicians to support their clinical work are mounting, 5 and other large language models dedicated to medical content are improving continuously. 6 Recently, ChatGPT has been updated with GPT-4V(ision), which can analyze uploaded images.…”

mentioning

confidence: 99%

Keeping Up With ChatGPT

Rogasch,

Jochens,

Metzger

et al. 2024

Clin Nucl Med

View full text Add to dashboard Cite

Purpose The latest iteration of GPT4 (generative pretrained transformer) is a large multimodal model that can integrate both text and image input, but its performance with medical images has not been systematically evaluated. We studied whether ChatGPT with GPT-4V(ision) can recognize images from common nuclear medicine examinations and interpret them. Patients and Methods Fifteen representative images (scintigraphy, 11; PET, 4) were submitted to ChatGPT with GPT-4V(ision), both in its Default and “Advanced Data Analysis (beta)” version. ChatGPT was asked to name the type of examination and tracer, explain the findings and whether there are abnormalities. ChatGPT should also mark anatomical structures or pathological findings. The appropriateness of the responses was rated by 3 nuclear medicine physicians. Results The Default version identified the examination and the tracer correctly in the majority of the 15 cases (60% or 53%) and gave an “appropriate” description of the findings or abnormalities in 47% or 33% of cases, respectively. The Default version cannot manipulate images. “Advanced Data Analysis (beta)” failed in all tasks in >90% of cases. A “major” or “incompatible” inconsistency between 3 trials of the same prompt was observed in 73% (Default version) or 87% of cases (“Advanced Data Analysis (beta)” version). Conclusions Although GPT-4V(ision) demonstrates preliminary capabilities in analyzing nuclear medicine images, it exhibits significant limitations, particularly in its reliability (ie, correctness, predictability, and consistency).

show abstract

“…The Viewpoint titled “The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models,” published on July 25, 2023, was corrected to fix the Funding/Support and Role of the Funder/Sponsor sections. “Dr Minssen also received support from the Horizon Europe project CLASSICA (grant 101057321)” was added to the Funding/Support section, and “the Horizon Europe project CLASSICA” was added to the Role of the Funder/Sponsor section.…”

mentioning

confidence: 99%

Additions to Funding/Support and Role of the Funder/Sponsor Sections

2023

JAMA

View full text Add to dashboard Cite

The Challenges for Regulating Medical Use of ChatGPT and Other Large Language Models

Abstract: This Viewpoint discusses how regulators across the world should approach the legal and ethical challenges, including privacy, device regulation, competition, intellectual property rights, cybersecurity, and liability, raised by the medical use of large language models.

Cited by 90 publications

References 5 publications

Large Language Models in Neurology Research and Future Practice

Large Language Models in Neurology Research and Future Practice

Keeping Up With ChatGPT

Additions to Funding/Support and Role of the Funder/Sponsor Sections

Contact Info

Product

Resources

About