Drug–drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths

Zhang, Yijia; Zheng, Wei; Lin, Hongfei; Wang, Jian; Yang, Zhihao; Dumontier, Michel

doi:10.1093/bioinformatics/btx659

Cited by 145 publications

(107 citation statements)

References 26 publications

(41 reference statements)

Supporting

Mentioning

106

Contrasting

Order By: Relevance

“…Afterward, we concatenated two sentence embeddings and fed them into an architecture with one dense layer to predict the similarity of two sentences. ; BC5CDR-disease, BC5CDR-chem (Yoon et al, 2018); ShARe/CLEFE (Leaman et al, 2015); DDI (Zhang et al, 2018). Chem-Prot (Peng et al, 2018); i2b2 (Rink et al, 2011); HoC (Du et al, 2019); MedNLI (Romanov and Shivade, 2018).…”

Section: Fine-tuning With Elmomentioning

confidence: 99%

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Peng

Yan

2019

Proceedings of the 18th BioNLP Workshop and Shared Task

614

462

View full text Add to dashboard Cite

Inspired by the success of the General Language Understanding Evaluation benchmark, we introduce the Biomedical Language Understanding Evaluation (BLUE) benchmark to facilitate research in the development of pre-training language representations in the biomedicine domain. The benchmark consists of five tasks with ten datasets that cover both biomedical and clinical texts with different dataset sizes and difficulties. We also evaluate several baselines based on BERT and ELMo and find that the BERT model pre-trained on PubMed abstracts and MIMIC-III clinical notes achieves the best results. We make the datasets, pre-trained models, and codes publicly available at https://github.com/ ncbi-nlp/BLUE_Benchmark.

show abstract

Section: Fine-tuning With Elmomentioning

confidence: 99%

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Peng

Yan

2019

Proceedings of the 18th BioNLP Workshop and Shared Task

614

462

View full text Add to dashboard Cite

show abstract

“…We ran the model 5 times with different random seeds and then calculated the average performance [62]. The state-of-the-art (SOTA) model by Zhang and colleagues achieved an F1-score of 0.73 on this dataset [63]. Their model uses an LSTM as an encoder with an attention mechanism and outperforms other feature-based, kernel-based, and neural networks-based methods.…”

Section: Extrinsic Evaluation Resultsmentioning

confidence: 99%

BioConceptVec: Creating and evaluating literature-based biomedical concept embeddings on a large scale

et al. 2020

View full text Add to dashboard Cite

A massive number of biological entities, such as genes and mutations, are mentioned in the biomedical literature. The capturing of the semantic relatedness of biological entities is vital to many biological applications, such as protein-protein interaction prediction and literature-based discovery. Concept embeddings-which involve the learning of vector representations of concepts using machine learning models-have been employed to capture the semantics of concepts. To develop concept embeddings, named-entity recognition (NER) tools are first used to identify and normalize concepts from the literature, and then different machine learning models are used to train the embeddings. Despite multiple attempts, existing biomedical concept embeddings generally suffer from suboptimal NER tools, small-scale evaluation, and limited availability.In response, we employed high-performance machine learning-based NER tools for concept recognition and trained our concept embeddings, BioConceptVec, via four different machine learning models on ~30 million PubMed abstracts. BioConceptVec covers over 400,000 biomedical concepts mentioned in the literature and is of the largest among the publicly available biomedical concept embeddings to date. To evaluate the validity and utility of BioConceptVec, we respectively performed two intrinsic evaluations (identifying related concepts based on drug-gene and gene-gene interactions) and two extrinsic evaluations (protein-protein interaction prediction and drug-drug interaction extraction), collectively using over 25 million instances from nine independent datasets (17 million instances from six intrinsic evaluation tasks and 8 million instances from three extrinsic evaluation tasks), which is, by far, the most comprehensive to our best knowledge. The intrinsic evaluation results demonstrate that BioConceptVec consistently has, by a large margin, better performance than existing concept embeddings in identifying similar and related concepts. More importantly, the extrinsic evaluation results demonstrate that using BioConceptVec with advanced deep learning models can significantly improve performance in downstream bioinformatics studies and biomedical text-mining applications.Our BioConceptVec embeddings and benchmarking datasets are publicly available at https://github.com/ncbi-nlp/BioConceptVec. Author SummaryCapturing the semantics of related biological concepts, such as genes and mutations, is of significant importance to many research tasks in computational biology such as protein-protein interaction detection, gene-drug association prediction, and biomedical literature-based discovery. Here, we propose to leverage state-of-the-art text mining tools and machine learning models to learn the semantics via vector representations (aka. embeddings) of over 400,000 biological concepts mentioned in the entire PubMed abstracts. Our learned embeddings, namely BioConceptVec, can capture related concepts based on their surrounding contextual information in the literature, which is beyond exact term ...

show abstract

“…Attention mechanisms have recently been successfully applied to biomedical relation extraction tasks [14,18,30]. These attention networks are able to learn a vector of important weights for each word in a sentence to reflect its impact on the final result.…”

Section: Attention Mechanismsmentioning

confidence: 99%

“…Unlike the feature-based models, DL models demand less feature engineering because they can automatically learn useful features from training data. Examples of popular DL models that have successfully been applied for biomedical relation extraction include Convolutional Neural Networks (CNNs) [9,10,11,12] and Recurrent Neural Networks (RNNs) [13,14].…”

mentioning

confidence: 99%

Relation Extraction Between Bacteria and Biotopes from Biomedical Texts with Attention Mechanisms and Domain-Specific Contextual Representations

Jettakul

Wichadakul

Vateekul

2019

Preprint

View full text Add to dashboard Cite

The Bacteria Biotope (BB) task is biomedical relation extraction (RE) that aims to study the interaction between bacteria and their locations. This task is considered to pertain to fundamental knowledge in applied microbiology. Some previous investigations have used feature-based models; others have presented deep-learning-based models such as convolutional and recurrent neural networks used with the shortest dependency paths (SDPs). Although SDPs contain valuable and concise information, sections of significant information necessary to define bacterial location relationships are often neglected. In addition, the traditional word embedding used in previous studies may suffer from word ambiguation across linguistic contexts.Here, we present a deep learning model for biomedical RE. The model incorporates feature combinations of SDPs and full sentences with various attention mechanisms. We also used pre-trained contextual representations based on domain-specific vocabularies. In order to assess the model's robustness, we introduced a mean F1 score on many models using different random seeds. The experiments were conducted on the standard BB corpus in BioNLP-ST'16. Our experimental results revealed that the model performed better (in terms of both maximum and average F1 scores; 60.77% and 57.63%, respectively) compared with other existing models. We demonstrated that our proposed contributions to this task can be used to extract rich lexical, syntactic, and semantic features that effectively boost the model's performance. Moreover, we analyzed the trade-off between precision and recall in order to choose the proper cut-off to use in real-world applications.K eywords Biomedical text mining · Relation extraction · Deep learning · Attention networks · Contextual word embeddings · Domain-specific language IntroductionDue to the rapid development of computational and biological technology, the biomedical literature is expanding at an exponential rate [1]. This situation leads to difficulty manually extracting required information. In BioNLP-ST 2016, the Bacteria Biotope (BB) task [2] followed the general outline and goals of previous tasks defined in 2011 [3] and 2013 [4]. This task aims to investigate the interactions of bacteria and its biotope; habitats or geographical entity, from genetic, phylogenetic, and ecology perspectives. It involves the Lives_in relation, which is a mandatory relation between related arguments, the bacteria and the location * peerapon.v@chula.ac.th Our highest precision (0.975 cut-off) Recall: 46.90 Precision: 72.60 F1 score: 56.99 Our default (0.5 cut-off) Recall: 65.28 Precision: 56.85 F1 score: 60.77 Our highest recall (0.025 cut-off) Recall: 70.54 Precision: 50.11 F1 score: 58.59 BGRU-Attn Recall: 69.82 Precision: 48.76 F1 score: 57.42 TurkuNLP Recall: 44.80 Precision: 62.30 F1 score: 52.10

show abstract

Drug–drug interaction extraction via hierarchical RNNs on sequence and shortest dependency paths

Cited by 145 publications

References 26 publications

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

Transfer Learning in Biomedical Natural Language Processing: An Evaluation of BERT and ELMo on Ten Benchmarking Datasets

BioConceptVec: Creating and evaluating literature-based biomedical concept embeddings on a large scale

Relation Extraction Between Bacteria and Biotopes from Biomedical Texts with Attention Mechanisms and Domain-Specific Contextual Representations

Contact Info

Product

Resources

About