Leveraging pre-trained language models for mining microbiome-disease relationships

Abstract

Background The growing recognition of the microbiome’s impact on human health and well-being has prompted extensive research into discovering the links between microbiome dysbiosis and disease (healthy) states. However, this valuable information is scattered in unstructured form within biomedical literature. The structured extraction and qualification of microbe-disease interactions are important. In parallel, recent advancements in deep-learning-based natural language processing algorithms have revolutionized language-related tasks such as ours. This study aims to leverage state-of-the-art deep-learning language models to extract microbe-disease relationships from biomedical literature. Results In this study, we first evaluate multiple pre-trained large language models within a zero-shot or few-shot learning context. In this setting, the models performed poorly out of the box, emphasizing the need for domain-specific fine-tuning of these language models. Subsequently, we fine-tune multiple language models (specifically, GPT-3, BioGPT, BioMedLM, BERT, BioMegatron, PubMedBERT, BioClinicalBERT, and BioLinkBERT) using labeled training data and evaluate their performance. Our experimental results demonstrate the state-of-the-art performance of these fine-tuned models ( specifically GPT-3, BioMedLM, and BioLinkBERT), achieving an average F1 score, precision, and recall of over $$>0.8$$ > 0.8 compared to the previous best of 0.74. Conclusion Overall, this study establishes that pre-trained language models excel as transfer learners when fine-tuned with domain and problem-specific data, enabling them to achieve state-of-the-art results even with limited training data for extracting microbiome-disease interactions from scientific publications.

A Systematic Review of ChatGPT and Other Conversational Large Language Models in Healthcare

“…Beyond ChatGPT, other generative AI frameworks, such as DR.BENCH [62], were employed for clinical diagnostic reasoning tasks [62]. Moreover, various pre-trained LLMs can extract microbe-disease relationships from biomedical texts in zero-shot/few-shot contexts with high accuracy, with an average F1 score, precision, and recall greater than 80% [63]. In addition, ChatGPT was the best LLM when predicting high acuity cases than predicting low acuity cases according to emergency severity index (ESI), with a sensitivity of 76.2%, a specificity of 93.1%, compared to the overall sensitivity of 57.1%, specificity of 34.5% [64].…”

“…Since recent LLMs are trained based on human-generated texts from the Internet, they also tend to provide biased answers [4]. Besides, algorithms may reinforce current health disparities and inequities [63]. Indeed, outputs from ChatGPT have been shown to be biased in terms of gender, race, and religion [4].…”

Large Language Models in Oncology: Revolution or Cause for Concern?

“…The data extraction ability of LLMs can also be enhanced through fine-tuning. This includes pre-trained LLMs in the generative and discriminative setting, i.e., they can generate responses to a question when prompted in a given context and classify input data into predefined labels [59]. Domain-specific LLMs, such as BioMedLM and BioGPT, are trained with data from the biomedical literature on PubMed and can be fine-tuned with gold standard oncology corpora [60,61].…”

Designing Incremental Knowledge Enrichment in Generative Pre-trained Transformers

“…Validation processes have emphasized the importance of contextual relevance and applicability of the knowledge to specific tasks or domains [58,59,60,61]. Peer review mechanisms and consensus algorithms have been utilized to enhance the trustworthiness of collaboratively curated knowledge bases [62,61,63]. The application of cryptographic techniques for knowledge verification has provided a means to secure the integrity of information in distributed systems [64,65].…”