Multi-Labeled Relation Extraction with Attentive Capsule Network

Zhang, Xinsong; Li, Pengshuai; Jia, Weijia; Zhang, Hai

doi:10.1609/aaai.v33i01.33017484

Cited by 56 publications

(27 citation statements)

References 3 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They also found that capsule networks exhibit significant improvement when transferring single-label to multi-label text classification. Similar property has also been observed in the task of relation extraction (Zhang et al, 2019. However, interactive word-level attention is not considered in these typical capsule routing methods.…”

Section: Capsule Networksupporting

confidence: 67%

“…Capsule network (Hinton et al, 2011;Sabour et al, 2017;Hinton et al, 2018) constructs vectorbased feature representation. Capsules in adjacent layers are connected by dynamic routing, which shows strengths in distinguishing overlapped features by feature clustering (Sabour et al, 2017;Zhang et al, 2019). In the aspect-level sentimental classification task, the vector-based overlapped sentimental features towards different aspect terms will be clustered by an Expectation-Maximization (EM) routing algorithm, which makes the subsequent classification more clear.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Capsule Network with Interactive Attention for Aspect-Level Sentiment Classification

Sun

Wang

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

Aspect-level sentiment classification is a crucial task for sentiment analysis, which aims to identify the sentiment polarities of specific targets in their context. The main challenge comes from multi-aspect sentences, which express multiple sentiment polarities towards different targets, resulting in overlapped feature representation. However, most existing neural models tend to utilize static pooling operation or attention mechanism to identify sentimental words, which therefore insufficient for dealing with overlapped features. To solve this problem, we propose to utilize capsule network to construct vector-based feature representation and cluster features by an EM routing algorithm. Furthermore, interactive attention mechanism is introduced in the capsule routing procedure to model the semantic relationship between aspect terms and context. The iterative routing also enables encoding sentence from a global perspective. Experimental results on three datasets show that our proposed model achieves state-of-the-art performance.

show abstract

Section: Capsule Networksupporting

confidence: 67%

Section: Introductionmentioning

confidence: 99%

Capsule Network with Interactive Attention for Aspect-Level Sentiment Classification

Sun

Wang

et al. 2019

Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conferen

View full text Add to dashboard Cite

show abstract

“…To reduce the impact of noisy words, tree-based methods attempt to obtain the relevant sub-structure of an instance for relation extraction (Xu et al, 2015;Miwa and Bansal, 2016;Liu et al, 2018). To discriminate overlapped relation features, (Zhang et al, 2019) apply the capsule network (Sabour et al, 2017) for multi-labeled relation extraction. Inspired by the ability of multi-head attention in modeling the long-term dependency (Vaswani et al, 2017), (Zhang et al, 2020) attempt to reduce multi-granularity noise via multi-head attention in relation extraction.…”

Section: Related Workmentioning

confidence: 99%

“…As shown in Figure 1 (Zeng et al, 2014) or self-attention (Lin et al, 2016). Although (Zhang et al, 2019) first propose an attentive capsule network for multi-labeled relation extraction, it treats the CNN/RNN as low-level capsules without the diversity encouragement, which poses the difficulty of distinguishing different and overlapped relation features from a single type of semantic capsule. Therefore, a well-behaved relation extractor is needed to discriminate diverse overlapped relation features from different semantic spaces.…”

Section: Introductionmentioning

confidence: 99%

Regularized Attentive Capsule Network for Overlapped Relation Extraction

Liu¹,

Lin²,

Jia³

et al. 2020

Proceedings of the 28th International Conference on Computational Linguistics

Self Cite

View full text Add to dashboard Cite

Distantly supervised relation extraction has been widely applied in knowledge base construction due to its less requirement of human efforts. However, the automatically established training datasets in distant supervision contain low-quality instances with noisy words and overlapped relations, introducing great challenges to the accurate extraction of relations. To address this problem, we propose a novel Regularized Attentive Capsule Network (RA-CapNet) to better identify highly overlapped relations in each informal sentence. To discover multiple relation features in an instance, we embed multi-head attention into the capsule network as the low-level capsules, where the subtraction of two entities acts as a new form of relation query to select salient features regardless of their positions. To further discriminate overlapped relation features, we devise disagreement regularization to explicitly encourage the diversity among both multiple attention heads and low-level capsules. Extensive experiments conducted on widely used datasets show that our model achieves significant improvements in relation extraction.

show abstract

“…They treated aggregation as a routing problem so that complex and interleaved features could be identified by the capsule network. Zhang et al [28] proposed capsule networks with dynamic routing for multi-labelled relation extraction to address the challenge of having multiple relations in a sentence. To better cluster features, they devised a dynamic routing algorithm based on an attention mechanism.…”

Section: Capsule Network For Relation Extractionmentioning

confidence: 99%

Capsule Networks With Word-Attention Dynamic Routing for Cultural Relics Relation Extraction

Zhang

Geng

2020

IEEE Access

View full text Add to dashboard Cite

Online museums and online cultural relic information provide abundant data for relation extraction research. However, in the relation extraction task of modelling space information, spatially insensitive methods of convolutional neural networks and long short term memory network in most current works still remain challenging in rich text structures, which makes models difficult to encode effectively and lacks the ability of text expression. To address this issue, we propose a framework named WAtt-Capsnet (the capsule network with word-attention dynamic routing), which is based on capsule networks with word-attention dynamic routing for the relation extraction task of online cultural relic data for capturing richer instantiation features. We further present combination embedding for capturing the characteristic information of Chinese sentences by considering the contribution of word embedding, parts of speech, character embedding and the position of words to capture rich internal structure information of sentences. More importantly, to reduce the decay of useful information in long sentences, we propose a routing algorithm based on a word-attention mechanism to focus on informative words. The experimental results demonstrate that the proposed method achieves significant performance for the relation extraction task of online cultural relic data.

show abstract

Multi-Labeled Relation Extraction with Attentive Capsule Network

Cited by 56 publications

References 3 publications

Capsule Network with Interactive Attention for Aspect-Level Sentiment Classification

Capsule Network with Interactive Attention for Aspect-Level Sentiment Classification

Regularized Attentive Capsule Network for Overlapped Relation Extraction

Capsule Networks With Word-Attention Dynamic Routing for Cultural Relics Relation Extraction

Contact Info

Product

Resources

About