Evaluating large language models as agents in the clinic

Mehandru, Nikita; Miao, Brenda Y.; Almaraz, Eduardo Rodriguez; Sushil, Madhumita; Butte, Atul J.; Alaa, Ahmed

doi:10.1038/s41746-024-01083-y

Cited by 15 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A significant focus has been on LLM applications for medical question answering and clinical reasoning, as well as agentic AI that assists clinicians with workflow tasks, such as summarising notes and medical documentation. [19][20][21] While VLMs using Gemini have been tested for interpreting OCT pathology, their reliability remains unproven. 22 In contrast, LVMs like…”

Section: History and Overview Of Generative Modelsmentioning

confidence: 99%

Generative artificial intelligence in ophthalmology: current innovations, future applications and challenges

Sonmez,

Sevgi,

Antaki

et al. 2024

Br J Ophthalmol

View full text Add to dashboard Cite

The rapid advancements in generative artificial intelligence are set to significantly influence the medical sector, particularly ophthalmology. Generative adversarial networks and diffusion models enable the creation of synthetic images, aiding the development of deep learning models tailored for specific imaging tasks. Additionally, the advent of multimodal foundational models, capable of generating images, text and videos, presents a broad spectrum of applications within ophthalmology. These range from enhancing diagnostic accuracy to improving patient education and training healthcare professionals. Despite the promising potential, this area of technology is still in its infancy, and there are several challenges to be addressed, including data bias, safety concerns and the practical implementation of these technologies in clinical settings.

show abstract

Section: History and Overview Of Generative Modelsmentioning

confidence: 99%

Generative artificial intelligence in ophthalmology: current innovations, future applications and challenges

Sonmez,

Sevgi,

Antaki

et al. 2024

Br J Ophthalmol

View full text Add to dashboard Cite

show abstract

“…It is necessary to carry out more accurate assessment of LLMs’ functionality in real-world clinical scenarios. [4]…”

Section: Introductionmentioning

confidence: 99%

“…It is necessary to carry out more accurate assessment of LLMs' functionality in real-world clinical scenarios. [4] Recent studies highlight the escalating global diabetes burden, advocating for innovative management strategies like the application of large language models (LLMs), including GPT-4. These models offer promising advances in diabetes care by potentially enhancing guideline adherence and providing personalized, evidencebased treatment recommendations.…”

Section: Introductionmentioning

confidence: 99%

Evaluating the Effectiveness and Safety of Large Language Model in Generating Type 2 Diabetes Mellitus Management Plans: A Comparative Study with Medical Experts Based on Real Patient Records

Mondal,

Naskar,

Roy Choudhury

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: The integration of large language models (LLMs) such as GPT-4 into healthcare presents potential benefits and challenges. While LLMs have shown promise in applications ranging from scientific writing to personalized medicine, their practical utility and safety in clinical settings remain under scrutiny. Concerns about accuracy, ethical considerations and bias necessitate rigorous evaluation of these technologies against established medical standards. Objective: To compare the completeness, necessity, dosage accuracy and overall safety of type 2 diabetes management plans created by GPT-4 with those devised by medical experts. Methods: This study involved a comparative analysis using anonymized patient records from a healthcare setting in West Bengal, India. Management plans for 50 Type 2 diabetes patients were generated by GPT-4 and three blinded medical experts. These plans were evaluated against a reference management plan based on American Diabetes Society guidelines. Completeness, necessity and dosage accuracy were quantified and an error score was devised to assess the quality of the generated management plans. The safety of the management plans generated by GPT-4 was also assessed. Results: Results indicated that medical experts' management plans had fewer missing medications compared to those generated by GPT-4 (p=0.008). However, GPT-4 generated management plans included fewer unnecessary medications (p=0.003). No significant difference was observed in the accuracy of drug dosages (p=0.975). The overall error scores were comparable between human experts and GPT-4 (p=0.301). Safety issues were noted in 16% of the plans generated by GPT-4, highlighting potential risks associated with AI-generated management plans. Conclusion: The study demonstrates that while GPT-4 can effectively reduce unnecessary drug prescriptions, it does not yet match the performance of medical experts in terms of plan completeness and safety. The findings support the use of LLMs as supplementary tools in healthcare, underscoring the need for enhanced algorithms and continuous human oversight to ensure the efficacy and safety of AI applications in clinical settings. Further research is necessary to improve the integration of LLMs into complex healthcare environments.

show abstract

“…Furthermore, we evaluate the influence of temperature settings on the clinical reasoning capabilities of various LLMs when tasked with interpreting clinical data. This investigation serves to deepen our understanding of LLM functionality in healthcare environments [10][11] [12].…”

Section: Introductionmentioning

confidence: 99%

Exploring Temperature Effects on Large Language Models Across Various Clinical Tasks

Patel,

Timsina,

Raut

et al. 2024

Preprint

View full text Add to dashboard Cite

Large Language Models (LLMs) are becoming integral to healthcare analytics. However, the influence of the temperature hyperparameter, which controls output randomness, remains poorly understood in clinical tasks. This study evaluates the effects of different temperature settings across various clinical tasks. We conducted a retrospective cohort study using electronic health records from the Mount Sinai Health System, collecting a random sample of 1283 patients from January to December 2023. Three LLMs (GPT-4, GPT-3.5, and Llama-3-70b) were tested at five temperature settings (0.2, 0.4, 0.6, 0.8, 1.0) for their ability to predict in-hospital mortality (binary classification), length of stay (regression), and the accuracy of medical coding (clinical reasoning). For mortality prediction, all models’ accuracies were generally stable across different temperatures. Llama-3 showed the highest accuracy, around 90%, followed by GPT-4 (80-83%) and GPT-3.5 (74-76%). Regression analysis for predicting the length of stay showed that all models performed consistently across different temperatures. In the medical coding task, performance was also stable across temperatures, with GPT-4 achieving the highest accuracy at 17% for complete code accuracy. Our study demonstrates that LLMs maintain consistent accuracy across different temperature settings for varied clinical tasks, challenging the assumption that lower temperatures are necessary for clinical reasoning.

show abstract

Evaluating large language models as agents in the clinic

Cited by 15 publications

References 24 publications

Generative artificial intelligence in ophthalmology: current innovations, future applications and challenges

Generative artificial intelligence in ophthalmology: current innovations, future applications and challenges

Evaluating the Effectiveness and Safety of Large Language Model in Generating Type 2 Diabetes Mellitus Management Plans: A Comparative Study with Medical Experts Based on Real Patient Records

Exploring Temperature Effects on Large Language Models Across Various Clinical Tasks

Contact Info

Product

Resources

About