Fine-tuning the BERTSUMEXT model for Clinical Report Summarization

Vinod, Pooja; Safar, Seema; Mathew, Divins; Venugopal, P. Dilip; Joly, Linta Merin; George, Joish

doi:10.1109/incet49848.2020.9154087

Cited by 13 publications

(3 citation statements)

References 1 publication

Supporting

Mentioning

Contrasting

Order By: Relevance

“…LLMs have the potential to help psychiatrists and other healthcare professionals with routine tasks such as writing of clinical reports, saving time and reducing manual data management (Cheng et al, 2023 ; Javaid et al, 2023 ). They have been used to provide summaries of patient-doctor conversations (Zhang et al, 2021 ), clinical notes (Kanwal and Rizzo, 2022 ) and reports (Vinod et al, 2020 ), as well as coding adverse events in patient narratives (Chopard et al, 2021 ). Furthermore, although off-the-shelf LLMs lack the sophistication required to answer queries of medical experts, finetuned models such as PMC-Llama (Wu et al, 2023 ) and Med-PaLM (Singhal et al, 2023 ) show increased expertise.…”

Section: Methodsmentioning

confidence: 99%

Introduction to Large Language Models (LLMs) for dementia care and research

Treder,

Lee,

Tsvetanov

2024

Front. Dement.

View full text Add to dashboard Cite

IntroductionDementia is a progressive neurodegenerative disorder that affects cognitive abilities including memory, reasoning, and communication skills, leading to gradual decline in daily activities and social engagement. In light of the recent advent of Large Language Models (LLMs) such as ChatGPT, this paper aims to thoroughly analyse their potential applications and usefulness in dementia care and research.MethodTo this end, we offer an introduction into LLMs, outlining the key features, capabilities, limitations, potential risks, and practical considerations for deployment as easy-to-use software (e.g., smartphone apps). We then explore various domains related to dementia, identifying opportunities for LLMs to enhance understanding, diagnostics, and treatment, with a broader emphasis on improving patient care. For each domain, the specific contributions of LLMs are examined, such as their ability to engage users in meaningful conversations, deliver personalized support, and offer cognitive enrichment. Potential benefits encompass improved social interaction, enhanced cognitive functioning, increased emotional well-being, and reduced caregiver burden. The deployment of LLMs in caregiving frameworks also raises a number of concerns and considerations. These include privacy and safety concerns, the need for empirical validation, user-centered design, adaptation to the user's unique needs, and the integration of multimodal inputs to create more immersive and personalized experiences. Additionally, ethical guidelines and privacy protocols must be established to ensure responsible and ethical deployment of LLMs.ResultsWe report the results on a questionnaire filled in by people with dementia (PwD) and their supporters wherein we surveyed the usefulness of different application scenarios of LLMs as well as the features that LLM-powered apps should have. Both PwD and supporters were largely positive regarding the prospect of LLMs in care, although concerns were raised regarding bias, data privacy and transparency.DiscussionOverall, this review corroborates the promising utilization of LLMs to positively impact dementia care by boosting cognitive abilities, enriching social interaction, and supporting caregivers. The findings underscore the importance of further research and development in this field to fully harness the benefits of LLMs and maximize their potential for improving the lives of individuals living with dementia.

show abstract

Section: Methodsmentioning

confidence: 99%

Introduction to Large Language Models (LLMs) for dementia care and research

Treder,

Lee,

Tsvetanov

2024

Front. Dement.

View full text Add to dashboard Cite

show abstract

“…In essence, the BERT model is based on an encoder-decoder network transformer. BERT has been adopted in the medical field as an extractive summarization approach, to summarise patients medical history [50]. Recently the other pretrained language model BART has attracted many researchers since it obtained new state-of-the-art performance in summarization task.…”

Section: Automatic Text Summarizationmentioning

confidence: 99%

DepressionNet: Learning Multi-modalities with User Post Summarization for Depression Detection on Social Media

Zogan

Razzak

Jameel

et al. 2021

Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

Twitter is currently a popular online social media platform which allows users to share their user-generated content. This publiclygenerated user data is also crucial to healthcare technologies because the discovered patterns would hugely benefit them in several ways. One of the applications is in automatically discovering mental health problems, e.g., depression. Previous studies to automatically detect a depressed user on online social media have largely relied upon the user behaviour and their linguistic patterns including user's social interactions. The downside is that these models are trained on several irrelevant content which might not be crucial towards detecting a depressed user. Besides, these content have a negative impact on the overall efficiency and effectiveness of the model. To overcome the shortcomings in the existing automatic depression detection methods, we propose a novel computational framework for automatic depression detection that initially selects relevant content through a hybrid extractive and abstractive summarization strategy on the sequence of all user tweets leading to a more fine-grained and relevant content. The content then goes to our novel deep learning framework comprising of a unified learning machinery comprising of Convolutional Neural Network (CNN) coupled with attention-enhanced Gated Recurrent Units (GRU) models leading to better empirical performance than existing strong baselines.

show abstract

“…A research paper deals with the NLP methods to predict medical specialties from the unstructured text notes of a university hospital [ 28 ]. Vinod, P., et al demonstrated a deep learning model from clinical text data [ 29 ]. Teng, F., et al developed a deep learning model to predict ICD codes from free text data [ 30 ].…”

Section: Introductionmentioning

confidence: 99%

Applying Deep Learning Model to Predict Diagnosis Code of Medical Records

et al. 2023

View full text Add to dashboard Cite

The International Classification of Diseases (ICD) code is a diagnostic classification standard that is frequently used as a referencing system in healthcare and insurance. However, it takes time and effort to find and use the right diagnosis code based on a patient’s medical records. In response, deep learning (DL) methods have been developed to assist physicians in the ICD coding process. Our findings propose a deep learning model that utilized clinical notes from medical records to predict ICD-10 codes. Our research used text-based medical data from the outpatient department (OPD) of a university hospital from January to December 2016. The dataset used clinical notes from five departments, and a total of 21,953 medical records were collected. Clinical notes consisted of a subjective component, objective component, assessment, plan (SOAP) notes, diagnosis code, and a drug list. The dataset was divided into two groups: 90% for training and 10% for test cases. We applied natural language processing (NLP) technique (word embedding, Word2Vector) to process the data. A deep learning-based convolutional neural network (CNN) model was created based on the information presented above. Three metrics (precision, recall, and F-score) were used to calculate the achievement of the deep learning CNN model. Clinically acceptable results were achieved through the deep learning model for five departments (precision: 0.53–0.96; recall: 0.85–0.99; and F-score: 0.65–0.98). With a precision of 0.95, a recall of 0.99, and an F-score of 0.98, the deep learning model performed the best in the department of cardiology. Our proposed CNN model significantly improved the prediction performance for an automated ICD-10 code prediction system based on prior clinical information. This CNN model could reduce the laborious task of manual coding and could assist physicians in making a better diagnosis.

show abstract

Fine-tuning the BERTSUMEXT model for Clinical Report Summarization

Cited by 13 publications

References 1 publication

Introduction to Large Language Models (LLMs) for dementia care and research

Introduction to Large Language Models (LLMs) for dementia care and research

DepressionNet: Learning Multi-modalities with User Post Summarization for Depression Detection on Social Media

Applying Deep Learning Model to Predict Diagnosis Code of Medical Records

Contact Info

Product

Resources

About