Natural Language Processing Model for Identifying Critical Findings—A Multi-Institutional Study

Banerjee, Imon; Davis, Melissa; Vey, Brianna L.; Mazaheri, Sina; Khan, Fiza; Zavaletta, Vaz; Gérard, R; Gichoya, Judy Wawira; Patel, Bhavik N.

doi:10.1007/s10278-022-00712-w

Cited by 10 publications

(6 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Similarly, the BioBERT model that was trained on the same input text as BERT supplemented with PubMed abstracts and full-text articles significantly outperforms BERT on biomedical named entity recognition, question answering and relation extraction [ 19 ]. Within the existing literature in the clinical domain, domain-specific models are shown to outperform fine-tuned general BERT models virtually every time a direct comparison takes place, such as for the aforementioned ClinicalBERT [ 11 , 16 , 18 , 20 – 22 ] and BioBERT [ 11 , 19 , 21 , 23 , 24 ].…”

Section: Introductionmentioning

confidence: 99%

Extracting patient lifestyle characteristics from Dutch clinical text with BERT models

Muizelaar,

Haas,

van Dortmont

et al. 2024

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

Background BERT models have seen widespread use on unstructured text within the clinical domain. However, little to no research has been conducted into classifying unstructured clinical notes on the basis of patient lifestyle indicators, especially in Dutch. This article aims to test the feasibility of deep BERT models on the task of patient lifestyle classification, as well as introducing an experimental framework that is easily reproducible in future research. Methods This study makes use of unstructured general patient text data from HagaZiekenhuis, a large hospital in The Netherlands. Over 148 000 notes were provided to us, which were each automatically labelled on the basis of the respective patients’ smoking, alcohol usage and drug usage statuses. In this paper we test feasibility of automatically assigning labels, and justify it using hand-labelled input. Ultimately, we compare macro F1-scores of string matching, SGD and several BERT models on the task of classifying smoking, alcohol and drug usage. We test Dutch BERT models and English models with translated input. Results We find that our further pre-trained MedRoBERTa.nl-HAGA model outperformed every other model on smoking (0.93) and drug usage (0.77). Interestingly, our ClinicalBERT model that was merely fine-tuned on translated text performed best on the alcohol task (0.80). In t-SNE visualisations, we show our MedRoBERTa.nl-HAGA model is the best model to differentiate between classes in the embedding space, explaining its superior classification performance. Conclusions We suggest MedRoBERTa.nl-HAGA to be used as a baseline in future research on Dutch free text patient lifestyle classification. We furthermore strongly suggest further exploring the application of translation to input text in non-English clinical BERT research, as we only translated a subset of the full set and yet achieved very promising results.

show abstract

Section: Introductionmentioning

confidence: 99%

Extracting patient lifestyle characteristics from Dutch clinical text with BERT models

Muizelaar,

Haas,

van Dortmont

et al. 2024

BMC Med Inform Decis Mak

View full text Add to dashboard Cite

show abstract

“…In Fig 1, we presented the overall framework for generating two versions of summary of findings documented within radiology reports which contains the three primary modules -(i) section segmentation; (ii) noisy data generation for layman summary; (iii) two-step large language model finetuning; and (iv) user evaluation to evaluate the quality of both technical and layman summaries. Section segmentation: We utilize previously developed NLP methods to parse the clinical history, imaging protocol, findings, and impression sections of the radiology reports using section segmentation based on the header [25]. To generalize the section segmentation across multiple institutions, we extracted all the variations of the headers using a similar word list generated by Word2Vec language model trained on 3M radiology reports from Emory University Hospital.…”

Section: Methodsmentioning

confidence: 99%

“…We utilize previously developed NLP methods to parse the clinical history, imaging protocol, findings, and impression sections of the radiology reports using section segmentation based on the header[25]. To generalize the section segmentation across multiple institutions, we extracted all the variations of the headers using a similar word list generated by Word2Vec language model trained on 3M radiology reports from Emory University Hospital.…”

Section: Methodsmentioning

confidence: 99%

Patient Centric Summarization of Radiology Findings using Large Language Models

Tariq,

Fathizadeh,

Ramaswamy

et al. 2024

Preprint

Self Cite

View full text Add to dashboard Cite

Objective. Develop automated AI models for patient-sensitive summarization of radiology reports. Level of medical education or socio-economic background of a patient may dictate their level of understanding of medical jargon. Inability to understand primary findings from a radiology report may lead to unnecessary anxiety among patients or result in missed follow up. Materials and Methods. Computed tomography exams of chest were selected as a use-case for this study. Approximately 7K chest CT reports were collected from Mayo Clinic Enterprise. Summarization model was built on the T5 large language model (LLM) as its text-to-text transfer architecture is intuitively suited for abstractive text summarization, resulting in a model size of ~0.77B. Noisy groundtruth for model training was collected by prompting LLaMA 13B model. Results. We recruited both experts (board-certified radiologists) and laymen to manually evaluate summaries generated by model. Model-generated summaries rarely missed information as marked by majority opinion of radiologists. Laymen indicated 63% improvement in their understanding by reading layman summaries generated by the model. Comparative study with zero-shot performance of LLaMA indicated that LLaMA hallucinated and missed information 3 and 4 times more often, respectively, than the proposed model. Discussion. The proposed patient-sensitive summarization model can generate summaries for radiology reports understandable by patients with vastly different levels of medical knowledge. In addition, task-specific training allows for more reliable performance compared to much larger off-the-shelf models. Conclusions. The proposed model could improve adherence to follow up treatment suggested by radiology reports by increasing level of understanding of these reports for patients.

show abstract

“…Similarly, the BioBERT model that was trained on the same input text as BERT supplemented with PubMed abstracts and full-text articles significantly outperforms BERT on biomedical named entity recognition, question answering and relation extraction [19]. Within the existing literature in the clinical domain, domain-specific models are shown to outperform fine-tuned general BERT models virtually every time a direct comparison takes place, such as for the aforementioned ClinicalBERT [11,16,18,[20][21][22] and BioBERT [11,19,21,23,24].…”

Section: Introductionmentioning

confidence: 94%

Extracting Patient Lifestyle Characteristics from Dutch Clinical Text with BERT Models

Muizelaar,

Haas,

Dortmont

et al. 2024

Preprint

View full text Add to dashboard Cite

Background: BERT models have seen widespread use on unstructured text within the clinical domain. However, little to no research has been conducted into classifying unstructured clinical notes on the basis of patient lifestyle indicators, especially in Dutch. This article aims to test the feasibility of deep BERT models on the task of patient lifestyle classification, as well as introducing an experimental framework that is easily reproducible in future research. Methods: This study makes use of unstructured general patient text data from HagaZiekenhuis, a large hospital in The Netherlands. Over 148 000 notes were provided to us, which were each automatically labelled on the basis of the respective patients’ smoking, alcohol usage and drug usage statuses. In this paper we test feasibility of automatically assigning labels, and justify it using hand-labelled input. Ultimately, we compare macro F1-scores of string matching, SGD and several BERT models on the task of classifying smoking, alcohol and drug usage. We test Dutch BERT models and English models with translated input. Results: We find that our further pre-trained MedRoBERTa.nl-HAGA model outperformed every other model on smoking (0.93) and drug usage (0.77). Interestingly, our ClinicalBERT model that was merely fine-tuned on translated text performed best on the alcohol task (0.80). In t-SNE visualisations, we show our MedRoBERTa.nl-HAGA model is the best model to differentiate between classes in the embedding space, explaining its superior classification performance. Conclusions: We suggest MedRoBERTa.nl-HAGA to be used as a baseline in future research on Dutch free text patient lifestyle classification. We furthermore strongly suggest further exploring the application of translation to input text in non-English clinical BERT research, as we only translated a subset of the full set and yet achieved very promising results.

show abstract

Natural Language Processing Model for Identifying Critical Findings—A Multi-Institutional Study

Cited by 10 publications

References 8 publications

Extracting patient lifestyle characteristics from Dutch clinical text with BERT models

Extracting patient lifestyle characteristics from Dutch clinical text with BERT models

Patient Centric Summarization of Radiology Findings using Large Language Models

Extracting Patient Lifestyle Characteristics from Dutch Clinical Text with BERT Models

Contact Info

Product

Resources

About