An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition

Luo, Ling; Yang, Zhihao; Yang, Pei; Zhang, Yin; Wang, Lei; Lin, Hongfei; Wang, Jian

doi:10.1093/bioinformatics/btx761

Cited by 359 publications

(227 citation statements)

References 16 publications

Supporting

Mentioning

194

Contrasting

Order By: Relevance

“…Most traditional deep-learning-based NER approaches (e.g., LSTM and Bi-LSTM models) are limited in their ability to address tagging inconsistency problems (Luo et al, 2017). Most traditional deep-learning-based NER approaches (e.g., LSTM and Bi-LSTM models) are limited in their ability to address tagging inconsistency problems (Luo et al, 2017).…”

Section: Attention-based Bi-lstm Modelmentioning

confidence: 99%

“…We describe an attention-based, Bi-LSTM model, for the GNER extraction task. Most traditional deep-learning-based NER approaches (e.g., LSTM and Bi-LSTM models) are limited in their ability to address tagging inconsistency problems (Luo et al, 2017). A natural approach for overcoming this issue is to apply an attention mechanism (Vaswani et al, 2017).…”

Section: Attention-based Bi-lstm Modelmentioning

confidence: 99%

See 1 more Smart Citation

GNER: A Generative Model for Geological Named Entity Recognition Without Labeled Data Using Deep Learning

Qiu

Xie

Tao

2019

Earth and Space Science

View full text Add to dashboard Cite

A variety of detailed data about geological topics and geoscience knowledge are buried in the geoscience literature and rarely used. Named entity recognition (NER) provides both opportunities and challenges to leverage this wealth of data in the geoscience literature for data analysis and further information extraction. Existing NER models and techniques are mainly based on rule‐based and supervised approaches, and developing such systems requires a costly manual effort. In this paper, we first design a generic stepwise framework for domain‐specific NER. Following this framework, domain‐specific entities and domain‐general words are collected and selected as seed terms. Normalization and grouping processes are then applied to these seed terms for further analysis. A random extraction algorithm based on a unigram language model is used to generate a large‐scale training data set consisting of probabilistically labeled pseudosentences. Each generated sentence is then used as input to the self‐training and learning algorithm. Experimental results on two constructed data sets demonstrate that the proposed model effectively recognizes and identifies geological named entities.

show abstract

Section: Attention-based Bi-lstm Modelmentioning

confidence: 99%

Section: Attention-based Bi-lstm Modelmentioning

confidence: 99%

GNER: A Generative Model for Geological Named Entity Recognition Without Labeled Data Using Deep Learning

Qiu

Xie

Tao

2019

Earth and Space Science

View full text Add to dashboard Cite

show abstract

“…Zhao et al [25] proposed a multiple label strategy (MLS) that can replace the CRF layer of a deep neural network for detecting spans of disease names. Korvigo et al [26] applied a CNN-RNN network to recognize spans of chemicals and Luo et al 2018 [28] proposed attention-based bidirectional LSTM with CRF to detect spans of chemicals. Unanue et al, 2017 [29] used bidirectional LSTM with CRF to detect spans of drug names and clinical concepts, while Lyu et al 2017 [27] proposed bidirectional LSTM-RNN model for detecting spans of a variety of biomedical concepts.…”

Section: Related Workmentioning

confidence: 99%

Biomedical Concept Recognition Using Deep Neural Sequence Models

Bada

et al. 2019

Preprint

View full text Add to dashboard Cite

Background: the automated identification of mentions of ontological concepts in natural language texts is a central task in biomedical information extraction. Despite more than a decade of effort, performance in this task remains below the level necessary for many applications.Results: recently, applications of deep learning in natural language processing have demonstrated striking improvements over previously state-of-the-art performance in many related natural language processing tasks. Here we demonstrate similarly striking performance improvements in recognizing biomedical ontology concepts in full text journal articles using deep learning techniques originally developed for machine translation. For example, our best performing system improves the performance of the previous state-of-the-art in recognizing terms in the Gene Ontology Biological Process hierarchy, from a previous best F1 score of 0.40 to an F1 of 0.70, nearly halving the error rate. Nearly all other ontologies show similar performance improvements. Conclusions:A two-stage concept recognition system, which is a conditional random field model for span detection followed by a deep neural sequence model for normalization, improves the state-of-the-art performance for biomedical concept recognition. Treating the biomedical concept normalization task as a sequence-tosequence mapping task similar to neural machine translation improves performance.

show abstract

“…All these approaches achieved remarkable performance over the traditional approaches to sequence labelling. To further improve the performance of neural models, other works [11]- [13] leverage additional information from linguistic features such as chunking and POS tags. However, the change in performance depends on the degree of semantic relatedness between the linguistic features and the target task.…”

Section: Related Workmentioning

confidence: 99%

Gated Task Interaction Framework for Multi-task Sequence Tagging

Ampomah

Lin

McClean

et al. 2019

2019 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Recent studies have shown that neural models can achieve high performance on several sequence labelling/tagging problems without the explicit use of linguistic features such as part-of-speech (POS) tags. These models are trained only using the character-level and the word embedding vectors as inputs. Others have shown that linguistic features can improve the performance of neural models on tasks such as chunking and named entity recognition (NER). However, the change in performance depends on the degree of semantic relatedness between the linguistic features and the target task; in some instances, linguistic features can have a negative impact on performance. This paper presents an approach to jointly learn these linguistic features along with the target sequence labelling tasks with a new multi-task learning (MTL) framework called Gated Tasks Interaction (GTI) network for solving multiple sequence tagging tasks. The GTI network exploits the relations between the multiple tasks via neural gate modules. These gate modules control the flow of information between the different tasks. Experiments on benchmark datasets for chunking and NER show that our framework outperforms other competitive baselines trained with and without external training resources.

show abstract

An attention-based BiLSTM-CRF approach to document-level chemical named entity recognition

Abstract: Supplementary data are available at Bioinformatics online.

Cited by 359 publications

References 16 publications

GNER: A Generative Model for Geological Named Entity Recognition Without Labeled Data Using Deep Learning

GNER: A Generative Model for Geological Named Entity Recognition Without Labeled Data Using Deep Learning

Biomedical Concept Recognition Using Deep Neural Sequence Models

Gated Task Interaction Framework for Multi-task Sequence Tagging

Contact Info

Product

Resources

About