What Does DALL-E 2 Know About Radiology? (Preprint)

Adams, Lisa C.; Busch, Felix; Truhn, Daniel; Makowski, Marcus R.; Aerts, Hugo JWL; Bressem, Keno K.

doi:10.2196/preprints.43110

Cited by 7 publications

(4 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Since the Large Language Models (LLMs) ChatGPT and GPT-4 were released in November 2022 and March 2023 respectively, multiple studies have shown how these LLMs could be applied to a variety of radiological tasks such as structured reporting, question answering on a radiology board-style examination, and response to common lung cancer questions (6)(7)(8). Closely related to our work, GPT-4 was shown to generate impressions for radiology reports (9).…”

Section: Introductionmentioning

confidence: 64%

An Open-source Fine-tuned Large Language Model for Radiological Impression Generation: A Multi-reader Performance Study

Serapio,

Chaudhari,

Savage

et al. 2024

Preprint

View full text Add to dashboard Cite

Background The impression section integrates key findings of a radiology report but can be subjective and variable. A fine-tuned open-source Large Language Model (LLM) was evaluated in its ability to generate radiological report impressions across different imaging modalities and hospitals. We sought to clinically validate an open-source fine-tuned LLM that automatically generates impressions to summarize radiology reports. Methods In this institutional review board-approved retrospective study, we fine-tuned an open-source LLM to generate the impression from the remainder of the radiology report. CT, US, and MRI radiology reports from Hospital 1 (n = 372716) and Hospital 2 (n = 60049), both under a single institution, were included in this study. The ROUGE score was used for automatic natural language evaluation and a reader study with five thoracic radiologists was performed for a clinical evaluation of CT chest impressions with a subspecialist baseline. We also stratified the results of the reader performance study based on the diagnosis category and the original impression length to gauge case complexity. Results The large language model achieved ROUGE-L scores of 46.51, 44.2, and 50.96 on the Hospital 1 dataset across the CT, US, and MRI modalities respectively. Upon external validation on the Hospital 2 independent test dataset, the model achieved ROUGE-L scores of 40.74, 37.89, and 24.61 for the same set of modalities. For the reader performance study, the model achieved overall mean scores of 3.56/4, 3.92/4, and 3.37/4, 18.29 seconds, and 12.32 words for clinical accuracy, grammatical accuracy, stylistic quality, edit time, and edit distance respectively. The LLM achieved the highest clinical accuracy ratings for acute/emergent findings. In terms of impression length, the LLM performed the best in clinical accuracy on shorter impressions. Conclusions We demonstrated that an open-source fine-tuned LLM can generate high-quality radiological impressions of clinical accuracy, grammatical accuracy, and stylistic quality across multiple imaging modalities and hospitals.

show abstract

Section: Introductionmentioning

confidence: 64%

An Open-source Fine-tuned Large Language Model for Radiological Impression Generation: A Multi-reader Performance Study

Serapio,

Chaudhari,

Savage

et al. 2024

Preprint

View full text Add to dashboard Cite

show abstract

“…However, they are mainly trained on textual datasets, and the safety issues exposed are limited to the NLP field. Recently, with the rise of multimodal pre-training models (e.g., NÜWA [153], DALL-E2 [110], GauGAN2 [119], etc. ), AI technology has demonstrated the powerful cross-modal (text-image) creation ability.…”

Section: Multimodal Information Processingmentioning

confidence: 99%

Recent Advances towards Safe, Responsible, and Moral Dialogue Systems: A Survey

Deng¹,

Sun²,

Zhang³

et al. 2023

Preprint

View full text Add to dashboard Cite

With the development of artificial intelligence, dialogue systems have been endowed with amazing chitchat capabilities, and there is widespread interest and discussion about whether the generated contents are socially beneficial. In this paper, we present a new perspective of research scope towards building a safe, responsible, and modal dialogue system, including 1) abusive and toxic contents, 2) unfairness and discrimination, 3) ethics and morality issues, and 4) risk of misleading and privacy information. Besides, we review the mainstream methods for evaluating the safety of large models from the perspectives of exposure and detection of safety issues. The recent advances in methodologies for the safety improvement of both end-to-end dialogue systems and pipeline-based models are further introduced. Finally, we discussed six existing challenges towards responsible AI: explainable safety monitoring, continuous learning of safety issues, robustness against malicious attacks, multimodal information processing, unified research framework, and multidisciplinary theory integration. We hope this survey will inspire further research toward safer dialogue systems. 1

show abstract

“…Numerous articles have highlighted several potential uses of GPT-4 in clinical medicine. These applications include creating virtual assistants in different kinds of surgery such as bariatric surgery, oral and maxillofacial surgery and colorectal surgery or converting unstructured radiology reports into structured formats [35][36][37][38].…”

Section: Applications Of Chatgpt and Gpt-4 In Medical And Dental Surgerymentioning

confidence: 99%

Applications of ChatGPT/GPT4 in Biological Studies, Medical and Dental Care

Bagheri Varzaneh

2023

Preprint

View full text Add to dashboard Cite

Chat generative pre-trained transformer (ChatGPT) is a developed language model and a subgroup of artificial intelligence (AI) which has demonstrated noticeable innovation in interactions between computer models and human studies. The release of ChatGPT in November 2022 attracted over 100 million users in a short time. It has unlimited applications in different fields of studies such as technology and science. ChatGPT utilizes deep learning and internet text to produce responses that resemble human language, although their accuracy is not always guaranteed. ChatGPT and GPT-4 can make a huge revolution in biology, medical, dental research, and health care. However, it's important to acknowledge the limitations of ChatGPT such as limitation in accessing the latest data. ChatGPT has generated both excitement and concern regarding its potential misuse. Its utilization in scientific publications has sparked debates and prompted the development of policies to govern the use of it. Although ChatGPT has certain limitations, it could impact on many different fields of study. There are challenges associated with using ChatGPT in the field of laboratory medicine and biology, particularly in the interpretation of test results. In this article, we review some of the applications of ChatGPT and GPT-4 in biology, dental and medical studies, and concerns about ChatGPT. Despite ChatGPT's conversational abilities are impressive, there are important considerations regarding its use in different fields of research and academic public.

show abstract

What Does DALL-E 2 Know About Radiology? (Preprint)

Cited by 7 publications

References 1 publication

An Open-source Fine-tuned Large Language Model for Radiological Impression Generation: A Multi-reader Performance Study

An Open-source Fine-tuned Large Language Model for Radiological Impression Generation: A Multi-reader Performance Study

Recent Advances towards Safe, Responsible, and Moral Dialogue Systems: A Survey

Applications of ChatGPT/GPT4 in Biological Studies, Medical and Dental Care

Contact Info

Product

Resources

About