Distant Supervision Relation Extraction with Intra-Bag and Inter-Bag Attentions

Abstract

This paper presents a neural relation extraction method to deal with the noisy training data generated by distant supervision. Previous studies mainly focus on sentence-level de-noising by designing neural networks with intra-bag attentions. In this paper, both intrabag and inter-bag attentions are considered in order to deal with the noise at sentence-level and bag-level respectively. First, relationaware bag representations are calculated by weighting sentence embeddings using intrabag attentions. Here, each possible relation is utilized as the query for attention calculation instead of only using the target relation in conventional methods. Furthermore, the representation of a group of bags in the training set which share the same relation label is calculated by weighting bag representations using a similarity-based inter-bag attention module. Finally, a bag group is utilized as a training sample when building our relation extractor. Experimental results on the New York Times dataset demonstrate the effectiveness of our proposed intra-bag and inter-bag attention modules. Our method also achieves better relation extraction accuracy than state-of-the-art methods on this dataset 1 .

Self-Attention Enhanced Selective Gate with Entity-Aware Embedding for Distantly Supervised Relation Extraction

“…In particular, instead of generating a sentence-level label, this framework assigns a label to a bag of sentences containing a common entity pair, and the label is a relationship of the entity pair on knowledge graph. Recently, based on the labeled data at bag level, a line of works (Zeng et al 2015;Du et al 2018;Lin et al 2016;Han et al 2018;Ye and Ling 2019) under selective attention framework (Lin et al 2016) let model implicitly focus on the correctly labeled sentence(s) by an attention mechanism and thus learn a stable and robust model from the noisy data.…”

“…Model comparison regarding the AUC value. The comparative results are reported byHan et al (2018) andYe and Ling (2019) respectively.…”

Improving Distantly-Supervised Relation Extraction Through BERT-Based Label and Instance Embeddings

“…Initially, a CNN-based method was proposed by [24] to automatically capture the semantics of sentences, while PCNN [25] became the common architecture to embed sentences. PCNN is used in several approaches that handle DS noisy patterns, such as intra-bag attention [5], inter-bag attention [26], soft labeling [27], [28] and adversarial training [29], [30]. Moreover, Graph-CNNs proved an effective way to encode syntactic information from text [8].…”

Data Augmentation by Rubrics for Short Answer Grading

“…The assumption is relaxed by (Riedel et al 2010) as expressed-at-least-once assumption, saying that if two entities participate in a relation, at least one sentence that mentions these two entities might express that relation. To augment the dataset for the relation classification task, (Ye and Ling 2019) found all sentences containing those entities for each pair of entities that appear in some Freebase relation in a large unlabeled corpus extract textual features to train the classifier. (Wei and Zou 2019) proposed an Easy Data Augmentation (EDA) method.…”