Biomedical event extraction based on GRU integrating attention mechanism

Abstract

BackgroundBiomedical event extraction is a crucial task in biomedical text mining. As the primary forum for international evaluation of different biomedical event extraction technologies, BioNLP Shared Task represents a trend in biomedical text mining toward fine-grained information extraction (IE). The fourth series of BioNLP Shared Task in 2016 (BioNLP-ST’16) proposed three tasks, in which the Bacteria Biotope event extraction (BB) task has been put forward in the earlier BioNLP-ST. Deep learning methods provide an effective way to automatically extract more complex features and achieve notable results in various natural language processing tasks.ResultsThe experimental results show that the presented approach can achieve an F-score of 57.42% in the test set, which outperforms previous state-of-the-art official submissions to BioNLP-ST 2016.ConclusionsIn this paper, we propose a novel Gated Recurrent Unit Networks framework integrating attention mechanism for extracting biomedical events between biotope and bacteria from biomedical literature, utilizing the corpus from the BioNLP’16 Shared Task on Bacteria Biotope task. The experimental results demonstrate the potential and effectiveness of the proposed framework.

Relation Extraction Between Bacteria and Biotopes from Biomedical Texts with Attention Mechanisms and Domain-Specific Contextual Representations

“…While the dependency paths only connect a single word to others within the same sentence (intra-sentence), there are some cross-sentence (inter-sentence) associations that can be very challenging in terms of the extraction task. In order to compare with other existing works [5,15,16,17,18], only intra-sentence relations were considered.…”

“…DET-BLSTM [17] applied bidirectional LSTM (BLSTM) with a dynamic extended tree (DET) adapted from SDPs and achieved an F1 score of 57.14%. Recently, BGRU-Attn [18] proposed bidirectional gated recurrent unit (BGRU) with attention mechanism and domain-oriented distributed word representation. Consequently, it became the state-of-the-art DL system without hand-designed features for the BB task with an F1 score of 57.42%.…”

“…Despite the success of DL in previous studies, there are still several limitations to be considered. Although SDPs have been shown to contain valuable syntactic features for relation extraction [16,17,18,19,20,21], they still may miss some important information. For example, in Figure 1, the word "in", which should play a key role in defining the relation between the bacteria "mycobacteria" and the biotope "Queensland" is not included in SDP (represented by blue lines) because there is no dependency path between "in" and any entities.…”

“…Instead of learning from full sentences, attention networks have demonstrated success in a wide range of NLP tasks [25,26,27,28,29,30,31]. In addition, BGRU-Attn [18] first used the Additive attention mechanism [29] for the BB task to focus on only sections of the output from RNN instead of the entire outputs and achieved state-of-the-art performance. Other attention techniques such as Entity-Oriented attention [30] and Multi-Head attention [31] still have not been explored for this task.…”

“…We also apply the CNN to capture SDP features that are shorter, more valuable, and more concise. In addition, because attention mechanisms have been proven to be helpful in the BB task [18], we incorporate several kinds of attention mechanisms-Additive attention, Entity-Oriented attention, and Multi-Head attention-into the model. Furthermore, we integrate domain-specific contextual word representation into the model to provide word-sense disambiguation.…”