Decomposing word embedding with the capsule network

Liu, Xin; Chen, Qingcai; Liu, Yan; Siebert, Joanna; Hu, Baotian; Wu, Xiang-Ping; Tang, Buzhou

doi:10.1016/j.knosys.2020.106611

Cited by 7 publications

(5 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Pre-trained sentence embedding or word embedding encode the model input into the embedding vector. The experiments are performed with RoBERTa [54] models, XLNET [73], Albert [74], DistilBERT [72], and BilSTM [71] using word2vec (w2v) [85], Sigmoid [86], GLU [87], and global vector (GloVe) [88] embedding. A sentence encoder builds a 768-dimensional hidden layer for individual phrase representation.…”

Section: Hyper-parametersmentioning

confidence: 99%

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

et al. 2023

View full text Add to dashboard Cite

Social media, fake news, and different propaganda strategies have all contributed to an increase in misinformation online during the past ten years. As a result of the scarcity of high-quality data, the present datasets cannot be used to train a deep-learning model, making it impossible to establish an identification. We used a natural language processing approach to the issue in order to create a system that uses deep learning to automatically identify propaganda in news items. To assist the scholarly community in identifying propaganda in text news, this study suggested the propaganda texts (ProText) library. Truthfulness labels are assigned to ProText repositories after being manually and automatically verified with fact-checking methods. Additionally, this study proposed using a fine-tuned Robustly Optimized BERT Pre-training Approach (RoBERTa) and word embedding using multi-label multi-class text classification. Through experimentation and comparative research analysis, we address critical issues and collaborate to discover answers. We achieved an evaluation performance accuracy of 90%, 75%, 68%, and 65% on ProText, PTC, TSHP-17, and Qprop, respectively. The big-data method, particularly with deep-learning models, can assist us in filling out unsatisfactory big data in a novel text classification strategy. We urge collaboration to inspire researchers to acquire, exchange datasets, and develop a standard aimed at organizing, labeling, and fact-checking.

show abstract

Section: Hyper-parametersmentioning

confidence: 99%

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

et al. 2023

View full text Add to dashboard Cite

show abstract

“…However, that does not properly discriminate different users, contrary to the fact that different users in a tweet propagation network may contribute differently to classifying the tweet. In our UMLARD, we introduce a capsule attention layer inspired by the recent success of capsule networks [62][63][64]. The Capsule network was first proposed in [62] and the main idea is to replace the scalar-output feature detectors in traditional neural networks with vector-output capsules, and train the model by the dynamic routing algorithm.…”

Section: Capsule Attention For User-level Feature Fusionmentioning

confidence: 99%

Multi-view learning with distinguishable feature fusion for rumor detection

Chen

Zhang

Trajcevski

et al. 2022

Knowledge-Based Systems

View full text Add to dashboard Cite

“…The method leverages contextual embeddings, glosses, and semantic networks to achieve the full coverage. A more recent system [26] uses BERT to learn context embedding and the capsule network to decompose word embedding into multiple morpheme-like vectors.…”

Section: Related Workmentioning

confidence: 99%

“…It shows the ability of contextual representation and effectiveness of incorporating gloss knowledge. CapsDecE2S [26] utilizes capsule network to decompose the unsupervised word embedding into multiple morpheme-like vectors and merges them by contextual attention to generate context specific sense embedding. The CapsDecE2S and GlossBERT enable two strong baselines hard to beat.…”

mentioning

confidence: 99%

Capsule Network Improved Multi-Head Attention for Word Sense Disambiguation

2021

View full text Add to dashboard Cite

Word sense disambiguation (WSD) is one of the core problems in natural language processing (NLP), which is to map an ambiguous word to its correct meaning in a specific context. There has been a lively interest in incorporating sense definition (gloss) into neural networks in recent studies, which makes great contribution to improving the performance of WSD. However, disambiguating polysemes of rare senses is still hard. In this paper, while taking gloss into consideration, we further improve the performance of the WSD system from the perspective of semantic representation. We encode the context and sense glosses of the target polysemy independently using encoders with the same structure. To obtain a better presentation in each encoder, we leverage the capsule network to capture different important information contained in multi-head attention. We finally choose the gloss representation closest to the context representation of the target word as its correct sense. We do experiments on English all-words WSD task. Experimental results show that our method achieves good performance, especially having an inspiring effect on disambiguating words of rare senses.

show abstract

Decomposing word embedding with the capsule network

Cited by 7 publications

References 8 publications

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

Robust Benchmark for Propagandist Text Detection and Mining High-Quality Data

Multi-view learning with distinguishable feature fusion for rumor detection

Capsule Network Improved Multi-Head Attention for Word Sense Disambiguation

Contact Info

Product

Resources

About