A medical question answering system using large language models and knowledge graphs

Abstract

Question answering systems have become prominent in all areas, while in the medical domain it has been challenging because of the abundant domain knowledge. Retrieval based approach has become promising as large pretrained language models come forth. This study focuses on building a retrieval-based medical question answering system, tackling the challenge with large language models and knowledge extensions via graphs. We first retrieve an extensive but coarse set of answers via Elasticsearch efficiently. Then, we utilize semantic matching with pretrained language models to achieve a fine-grained ranking enhanced with named entity recognition and knowledge graphs to exploit the relation of the entities in question and answer. A new architecture based on siamese structures for answer selection is proposed. To evaluate the approach, we train and test the model on two Chinese data sets, NLPCC2017 and cMedQA. We also conduct experiments on two English data sets, TREC-QA and WikiQA. Our model achieves consistent improvement as compared to strong baselines on all data sets.Qualification studies with cMedQA and our in-house data set show that our system gains highly competitive performance. The proposed medical question answering system outperforms baseline models and systems in quantification and qualification evaluations.

Can large language models build causal graphs?

“…Furthermore, there has been some research investigating LLM's ability to answer and reason with medical text data. Several recent studies [Liévin et al, 2022, Guo et al, 2022 showed promising results on LLMs ability to answer medical exam questions. Others [Moradi et al, 2021, Gutiérrez et al, 2022 have shown that context-specific LLMs such as BioBert are able to outperform GPT-3 in medical domain NLP tasks.…”

Phenomics Assistant: An Interface for LLM-based Biomedical Knowledge Graph Exploration

“…In biomedical applications, some approaches utilize semantic similarity search via embedding vectors. Recent approaches include embedding search followed by neighborhood retrieval, filtering the associations by query similarity for LLM summarization [20], neighborhood filtering based on a query classification [21], additionally considering document collections [22], rewriting neighborhood descriptions in text [23], decomposing queries into logical constructs [24], fine-tuning model weights with additional KGsourced training data [25], and developing specialized graph neural networks for improved reasoning [26].…”

The new paradigm in machine learning – foundation models, large language models and beyond: a primer for physicians

“…65 Such augmented models can be further enhanced by using knowledge graphs (KGs) as input data: structured repositories of factual knowledge that efficiently capture complex contextual relationships between real-world facts and concepts. 66,67 These KGs can gather useful contextual information beyond isolated facts through relationships, perform more sophisticated semantic searches, provide explicit structured knowledge representation to improve accuracy, scale knowledge efficiently by expanding the graph versus the model and facilitate explainability by tracing facts back to their origins in the graph. However, they may be limited by being incomplete and not up to date with the latest medical knowledge.…”

“…Retrieval‐augmented generation (RAG) enhances LLM by using a pretrained retriever model to retrieve supplementary text from large external databases, including recent data, and seamlessly integrating this information into the model to generate more accurate, contextualised responses 65 . Such augmented models can be further enhanced by using knowledge graphs (KGs) as input data: structured repositories of factual knowledge that efficiently capture complex contextual relationships between real‐world facts and concepts 66,67 . These KGs can gather useful contextual information beyond isolated facts through relationships, perform more sophisticated semantic searches, provide explicit structured knowledge representation to improve accuracy, scale knowledge efficiently by expanding the graph versus the model and facilitate explainability by tracing facts back to their origins in the graph.…”