Preparing for the next pandemic via transfer learning from existing diseases with hierarchical multi-modal BERT: a study on COVID-19 outcome prediction

Agarwal, Khushbu; Choudhury, Sutanay; Tipirneni, Sindhu; Mukherjee, Pritam; Ham, Colby; Tamang, Suzanne; Baker, Matthew; Tang, Siyi; Kocaman, Veysel; Gevaert, Olivier; Ralló, Robert; Reddy, Chandan K.

doi:10.1038/s41598-022-13072-w

Cited by 14 publications

(8 citation statements)

References 42 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…BiLSTM-CNN-Char has also been used as the state-of-the-art and a conventional model for many biomedical datasets. However, the latter works show that adding the language models on the top of conventional models (e.g., BiLSTM) show good performance (38,39), which may be attributed to large scale pre-training tasks on the top.…”

Section: Resultsmentioning

confidence: 99%

Large-Scale Application of Named Entity Recognition to Biomedicine and Epidemiology

Raza

Reji²,

Shajan³

et al. 2022

Preprint

View full text Add to dashboard Cite

Background: Despite significant advancements in biomedical named entity recognition methods, the clinical application of these systems continues to face many challenges: (1) most of the methods are trained on a limited set of clinical entities; (2) these methods are heavily reliant on a large amount of data for both pretraining and prediction, making their use in production impractical; (3) they do not consider non-clinical entities, which are also related to patient's health, such as social, economic or demographic factors. Methods: In this paper, we develop Bio-Epidemiology-NER (https://pypi.org/project/Bio-Epidemiology-NER/) an open-source Python package for detecting biomedical named entities from the text. This approach is based on Transformer-based approach and trained on a dataset that is annotated with many named entities (medical, clinical, biomedical and epidemiological). This approach improves on previous efforts in three ways: (1) it recognizes many clinical entity types, such as medical risk factors, vital signs, drugs, and biological functions; (2) it is easily configurable, reusable and can scale up for training and inference; (3) it also considers non-clinical factors (age and gender, race and social history and so) that influence health outcomes. At a high level, it consists of the phases: preprocessing, data parsing, named entity recognition and named entities enhancement. Results: Experimental results show that our pipeline outperforms other methods on three benchmark datasets with macro-and micro average F1 scores around 90 percent and above.

show abstract

Section: Resultsmentioning

confidence: 99%

Large-Scale Application of Named Entity Recognition to Biomedicine and Epidemiology

Raza

Reji²,

Shajan³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…BiLSTM-CNN-Char has also been used as a state-of-the-art and conventional model for many biomedical datasets. However, the latter works show that adding the language models on top of traditional models (e.g., BiLSTM) shows good performance [ 42 , 43 ], which may be attributed to large-scale pre-training tasks on the top.…”

Section: Resultsmentioning

confidence: 99%

Large-scale application of named entity recognition to biomedicine and epidemiology

Raza

Reji²,

Shajan³

et al. 2022

PLOS Digit Health

View full text Add to dashboard Cite

Background Despite significant advancements in biomedical named entity recognition methods, the clinical application of these systems continues to face many challenges: (1) most of the methods are trained on a limited set of clinical entities; (2) these methods are heavily reliant on a large amount of data for both pre-training and prediction, making their use in production impractical; (3) they do not consider non-clinical entities, which are also related to patient’s health, such as social, economic or demographic factors. Methods In this paper, we develop Bio-Epidemiology-NER (https://pypi.org/project/Bio-Epidemiology-NER/) an open-source Python package for detecting biomedical named entities from the text. This approach is based on a Transformer-based system and trained on a dataset that is annotated with many named entities (medical, clinical, biomedical, and epidemiological). This approach improves on previous efforts in three ways: (1) it recognizes many clinical entity types, such as medical risk factors, vital signs, drugs, and biological functions; (2) it is easily configurable, reusable, and can scale up for training and inference; (3) it also considers non-clinical factors (age and gender, race and social history and so) that influence health outcomes. At a high level, it consists of the phases: pre-processing, data parsing, named entity recognition, and named entity enhancement. Results Experimental results show that our pipeline outperforms other methods on three benchmark datasets with macro-and micro average F1 scores around 90 percent and above. Conclusion This package is made publicly available for researchers, doctors, clinicians, and anyone to extract biomedical named entities from unstructured biomedical texts.

show abstract

“…Most of these studies were performed in developed countries, and the considered indicators generally included comorbidities, demographic factors, laboratory data and symptoms. Some models also predicted the severity or mortality by considering the genetic indicators or metabolomics ( 38 , 45 – 48 ). Image analysis approaches based on deep learning algorithms were also utilized in the field diagnosis and prognosis of COVID-19 patients using CT and radiographic images ( 11 – 16 ).…”

Section: Discussionmentioning

confidence: 99%

Unraveling complex relationships between COVID-19 risk factors using machine learning based models for predicting mortality of hospitalized patients and identification of high-risk group: a large retrospective study

Banoei,

Rafiepoor,

Zendehdel

et al. 2023

Front. Med.

View full text Add to dashboard Cite

BackgroundAt the end of 2019, the coronavirus disease 2019 (COVID-19) pandemic increased the hospital burden of COVID-19 caused by the SARS-Cov-2 and became the most significant health challenge for nations worldwide. The severity and high mortality of COVID-19 have been correlated with various demographic characteristics and clinical manifestations. Prediction of mortality rate, identification of risk factors, and classification of patients played a crucial role in managing COVID-19 patients. Our purpose was to develop machine learning (ML)-based models for the prediction of mortality and severity among patients with COVID-19. Identifying the most important predictors and unraveling their relationships by classification of patients to the low-, moderate- and high-risk groups might guide prioritizing treatment decisions and a better understanding of interactions between factors. A detailed evaluation of patient data is believed to be important since COVID-19 resurgence is underway in many countries.ResultsThe findings of this study revealed that the ML-based statistically inspired modification of the partial least square (SIMPLS) method could predict the in-hospital mortality among COVID-19 patients. The prediction model was developed using 19 predictors including clinical variables, comorbidities, and blood markers with moderate predictability (Q2 = 0.24) to separate survivors and non-survivors. Oxygen saturation level, loss of consciousness, and chronic kidney disease (CKD) were the top mortality predictors. Correlation analysis showed different correlation patterns among predictors for each non-survivor and survivor cohort separately. The main prediction model was verified using other ML-based analyses with a high area under the curve (AUC) (0.81−0.93) and specificity (0.94−0.99). The obtained data revealed that the mortality prediction model can be different for males and females with diverse predictors. Patients were classified into four clusters of mortality risk and identified the patients at the highest risk of mortality, which accentuated the most significant predictors correlating with mortality.ConclusionAn ML model for predicting mortality among hospitalized COVID-19 patients was developed considering the interactions between factors that may reduce the complexity of clinical decision-making processes. The most predictive factors related to patient mortality were identified by assessing and classifying patients into different groups based on their sex and mortality risk (low-, moderate-, and high-risk groups).

show abstract

Preparing for the next pandemic via transfer learning from existing diseases with hierarchical multi-modal BERT: a study on COVID-19 outcome prediction

Cited by 14 publications

References 42 publications

Large-Scale Application of Named Entity Recognition to Biomedicine and Epidemiology

Large-Scale Application of Named Entity Recognition to Biomedicine and Epidemiology

Large-scale application of named entity recognition to biomedicine and epidemiology

Unraveling complex relationships between COVID-19 risk factors using machine learning based models for predicting mortality of hospitalized patients and identification of high-risk group: a large retrospective study

Contact Info

Product

Resources

About