Discovering Drug-Target Interaction Knowledge from Biomedical Literature

Abstract

The Interaction between Drugs and Targets (DTI) in human body plays a crucial role in biomedical science and applications. As millions of papers come out every year in the biomedical domain, automatically discovering DTI knowledge from biomedical literature, which are usually triplets about drugs, targets and their interaction, becomes an urgent demand in the industry. Existing methods of discovering biological knowledge are mainly extractive approaches that often require detailed annotations (e.g., all mentions of biological entities, relations between every two entity mentions, etc.). However, it is difficult and costly to obtain sufficient annotations due to the requirement of expert knowledge from biomedical domains. To overcome these difficulties, we explore the first end-to-end solution for this task by using generative approaches. We regard the DTI triplets as a sequence and use a Transformer-based model to directly generate them without using the detailed annotations of entities and relations. Further, we propose a semi-supervised method, which leverages the aforementioned end-to-end model to filter unlabeled literature and label them. Experimental results show that our method significantly outperforms extractive baselines on DTI discovery. We also create a dataset, KD-DTI, to advance this task and will release it to the community.

BioGPT: generative pre-trained transformer for biomedical text generation and mining

“…Considering that BioGPT is pre-trained on massive natural language corpus, we convert the labels to sequences in natural language rather than the structured format using special tokens explored in other works [14,24,25]. In this way, our reformed labels are semantically smoother than using special tokens.…”

“…These methods may suffer from error accumulation caused by previous tagging process and laborious intermediate annotations (i.e., named entity recognition). Text generation methods reframe the task as a sequence-to-sequence learning task, by taking the text as the input sequence and the triplet as the target sequence and employing an encoder-decoder network to learn to generate the triplet from the text [42,43,14,24,25]. However, many joint extraction methods still require additional entity information [38,44].…”

“…However, many joint extraction methods still require additional entity information [38,44]. In this work, we focus on the end-toend relation extraction, which formulates the task as an text generation task that takes only the text as the input and generates the relational triplets in an end-to-end way without additional intermediate annotations [24,14,25].…”

“…Generative models [6,7] directly generate the answer sequence or the label words. Relation Extraction GLRE [23], REBEL [24], seq2rel [25] KD-DTI [14], BC5CDR [13], DDI [15] Question Answering QA-Net [26], LUKE [27], BERT [2], PubMedBERT [9], BioELECTRA [28], LinkBERT [29] PubMedQA [16], BioASQ [30,31] Document Classification BERT [2], BlueBERT [8], SciBERT [18], SPECTER [32], PubMedBERT [9], BioELECTRA [28], LinkBERT [29] HoC [17], SciDocs [32] Document Classification Document classification is to classify a document into predefined categories (single label or multi label). Recent works on biomedical document classification also leverage large pre-trained language models for understanding the text and predicting the label [8,9,28,29].…”

“…BioGPT follows the Transformer language model backbone, and is pre-trained on 15M PubMed abstracts from scratch. We apply BioGPT to six biomedical NLP tasks: end-to-end relation extraction on BC5CDR [13], KD-DTI [14] and DDI [15], question answering on PubMedQA [16], document classification on HoC [17], and text generation. To adapt to the downstream tasks, we carefully design and analyze the target sequence format and the prompt for better modeling the tasks.…”