Calling Attention to Passages for Biomedical Question Answering

Abstract

Question answering can be described as retrieving relevant information for questions expressed in natural language, possibly also generating a natural language answer. This paper presents a pipeline for document and passage retrieval for biomedical question answering built around a new variant of the DeepRank network model in which the recursive layer is replaced by a self-attention layer combined with a weighting mechanism. This adaptation halves the total number of parameters and makes the network more suited for identifying the relevant passages in each document. The overall retrieval system was evaluated on the BioASQ tasks 6 and 7, achieving similar retrieval performance when compared to more complex network architectures.

Benchmarking a transformer-FREE model for ad-hoc retrieval

“…Our neural interaction-based model is an enhancement of Almeida and Matos (2020a) and was already used in the two international competitions, namely BioASQ (Almeida and Matos, 2020b;Tsatsaronis et al, 2015) and TREC-Covid (Almeida and Matos, 2020c;Roberts et al, 2020). However, in order to keep this paper self-contained, we will now introduce its insight and architecture.…”

Overview of BioASQ 2020: The Eighth BioASQ Challenge on Large-Scale Biomedical Semantic Indexing and Question Answering

“…The team from the University of Aveiro, also participated in phase A with its "bioinfo" systems, which consists of a finetuned BM25 retrieval model based on ElasticSearch [14], followed by a neural reranking step. For the latter, they use an interaction-based model inspired on the DeepRank [33] architecture building upon previous versions of their system [2]. The focus of the improvements was on the sentence splitting strategy, on extracting of multiple relevance signals, and the independent contribution of each sentence for the final score.…”

Frugal neural reranking: evaluation on the Covid-19 literature

“…The adopted neural ranking model is inspired by the DeepRank (Pang et al, 2017) architecture and represents an enhancement of our previous work (Almeida and Matos, 2020b), with the following major differences: the passage position input, proposed on the original work, was dropped; the detection network and the measure network were simplified and now form the interaction network; the contributions of each passage to the final document score are now assumed to be independent, and hence the self-attention layer proposed in (Almeida and Matos, 2020b) was replaced. The updated architecture can be visualized in Figure 3.…”