Transferability of Natural Language Inference to Biomedical Question Answering

Abstract

Biomedical question answering (QA) is a challenging task due to the scarcity of data and the requirement of domain expertise. Pre-trained language models have been used to address these issues. Recently, learning relationships between sentence pairs has been proved to improve performance in general QA. In this paper, we focus on applying BioBERT to transfer the knowledge of natural language inference (NLI) to biomedical QA. We observe that BioBERT trained on the NLI dataset obtains better performance on Yes/No (+5.59%), Factoid (+0.53%), List type (+13.58%) questions compared to performance obtained in a previous challenge (BioASQ 7B Phase B). We present a sequential transfer learning method that significantly performed well in the 8 th BioASQ Challenge (Phase B). In sequential transfer learning, the order in which tasks are fine-tuned is important. We measure an unanswerable rate of the extractive QA setting when the formats of factoid and list type questions are converted to the format of the Stanford Question Answering Dataset (SQuAD).

Sequence Tagging for Biomedical Extractive Question Answering

“…Baselines Our baseline model from Yoon et al ( 2019) is a challenge-winning model (BioASQ 7b) and can be described as an expanded version of the BioBERT model that can answer list-type questions by adding additional post-processing steps to decide the number of answers, a thresholding approach, and a rule-based number of answers detection from question strings for such questions containing a number. On top of the previous model, Jeong et al (2020) further added an additional transfer learning step using natural language inference (NLI) datasets and achieved performance improvement over the preceding models, winning the BioASQ 8b challenge.…”

“…To the best of our knowledge, was the first attempt to use a neural network (NN) based MRC model to achieve first place for the EQA problems of the 5th BioASQ challenge Task b -Phase B. BioBERT ) is a BERT model trained on biomedical data and showed a large performance improvement over preceding models for the BioEQA. Yoon et al (2019) and Jeong et al (2020) won the 7th and 8th BioASQ challenges, respectively, using BioBERT as a core building block for the factoid, list and yes/no questions (Nentidis et al, 2020). The aforementioned NN-based models for BioEQA Yoon et al, 2019;Jeong et al, 2020) formulated the training objective of their models as a start-end token prediction.…”

“…Yoon et al (2019) and Jeong et al (2020) won the 7th and 8th BioASQ challenges, respectively, using BioBERT as a core building block for the factoid, list and yes/no questions (Nentidis et al, 2020). The aforementioned NN-based models for BioEQA Yoon et al, 2019;Jeong et al, 2020) formulated the training objective of their models as a start-end token prediction.…”

“…The "start-end span prediction" setting, which is common in existing general domain EQA and Biomedical EQA (BioEQA) models Yoon et al, 2019;Jeong et al, 2020). is a straightforward and effective approach for predicting single answer spans.…”

“…Moreover, previous works on list questions require complex post-processing steps to determine the number of predicted answers. As an example, Yoon et al (2019) and Jeong et al (2020) applied a fixed probability threshold strategy which needed to be learned using a validation dataset, and rulebased query processing to detect the number of answers existing in the question.…”