TDAM: A topic-dependent attention model for sentiment analysis

Pergola, Gabriele; Gui, Lin; He, Yulan

doi:10.1016/j.ipm.2019.102084

Cited by 50 publications

(28 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Disentangled representation learning Deep generative models learn the hidden semantics of text, of which many attempt to capture the independent latent factor to steer the generation of text in the context of NLP (Hu et al, 2017;Li et al, 2018a;Pergola et al, 2019;John et al, 2019;. The majority of the aforementioned work employs VAE to learn controllable factors, leading to the abundance of VAE-based models in disentangled representation learning (Higgins et al, 2017;Burgess et al, 2018;Chen et al, 2018).…”

Section: Related Workmentioning

confidence: 99%

Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media

Zhu¹,

Zheng²,

Pergola³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

Building models to detect vaccine attitudes on social media is challenging because of the composite, often intricate aspects involved, and the limited availability of annotated data. Existing approaches have relied heavily on supervised training that requires abundant annotations and pre-defined aspect categories. Instead, with the aim of leveraging the large amount of unannotated data now available on vaccination, we propose a novel semisupervised approach for vaccine attitude detection, called VADET. A variational autoencoding architecture based on language models is employed to learn from unlabelled data the topical information of the domain. Then, the model is fine-tuned with a few manually annotated examples of user attitudes. We validate the effectiveness of VADET on our annotated data and also on an existing vaccination corpus annotated with opinions on vaccines. Our results show that VADET is able to learn disentangled stance and aspect topics, and outperforms existing aspect-based sentiment analysis models on both stance detection and tweet clustering.

show abstract

Section: Related Workmentioning

confidence: 99%

Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media

Zhu¹,

Zheng²,

Pergola³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Additionally, there are many previous supervised topic models that directly incorporate the supervision information into the unsupervised versions, including both works on single-label learning [21,14,38,39,35,36,29,24,26] and multi-label classification [32,27,28,30,13,17,25,37,1]. These models have empirically achieved very competitive classification performance, however, they require labeled documents as inputs.…”

Section: Related Workmentioning

confidence: 99%

Weakly Supervised Prototype Topic Model with Discriminative Seed Words: Modifying the Category Prior by Self-exploring Supervised Signals

Wang¹,

Wang²,

Li³

et al. 2021

Preprint

View full text Add to dashboard Cite

Dataless text classification, i.e., a new paradigm of weakly supervised learning, refers to the task of learning with unlabeled documents and a few predefined representative words of categories, known as seed words. The recent generative dataless methods construct document-specific category priors by using seed word occurrences only, however, such category priors often contain very limited and even noisy supervised signals. To remedy this problem, in this paper we propose a novel formulation of category prior. First, for each document, we consider its label membership degree by not only counting seed word occurrences, but also using a novel prototype scheme, which captures pseudo-nearest neighboring categories. Second, for each label, we consider its frequency prior knowledge of the corpus, which is also a discriminative knowledge for classification. By incorporating the proposed category prior into the previous generative dataless method, we suggest a novel generative dataless method, namely Weakly Supervised Prototype Topic Model (Wsptm). The experimental results on real-world datasets demonstrate that Wsptm outperforms the existing baseline methods.

show abstract

“…Hierarchical topic models (Viegas et al, 2020) utilize relationships among the latent topics. Supervised topic models have been explored previously where the topic model is trained through human feedback (Kumar et al, 2019) or with a task specific network simultaneously such that topic extraction is guided through task labels (Pergola et al, 2019;Wang and Yang, 2020). Card et al (2018) leverages document metadata but without metadata their method is same as ProdLDA which is our baseline.…”

Section: Introductionmentioning

confidence: 99%

TAN-NTM: Topic Attention Networks for Neural Topic Modeling

Panwar¹,

Shailabh²,

Aggarwal³

et al. 2021

Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Confer

View full text Add to dashboard Cite

Topic models have been widely used to learn text representations and gain insight into document corpora. To perform topic discovery, most existing neural models either take document bag-of-words (BoW) or sequence of tokens as input followed by variational inference and BoW reconstruction to learn topic-word distribution. However, leveraging topic-word distribution for learning better features during document encoding has not been explored much. To this end, we develop a framework TAN-NTM, which processes document as a sequence of tokens through a LSTM whose contextual outputs are attended in a topic-aware manner. We propose a novel attention mechanism which factors in topic-word distribution to enable the model to attend on relevant words that convey topic related cues. The output of topic attention module is then used to carry out variational inference. We perform extensive ablations and experiments resulting in ∼ 9 -15 percentage improvement over score of existing SOTA topic models in NPMI coherence on several benchmark datasets -20Newsgroups, Yelp Review Polarity and AGNews. Further, we show that our method learns better latent document-topic features compared to existing topic models through improvement on two downstream tasks: document classification and topic guided keyphrase generation.

show abstract

TDAM: A topic-dependent attention model for sentiment analysis

Cited by 50 publications

References 23 publications

Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media

Disentangled Learning of Stance and Aspect Topics for Vaccine Attitude Detection in Social Media

Weakly Supervised Prototype Topic Model with Discriminative Seed Words: Modifying the Category Prior by Self-exploring Supervised Signals

TAN-NTM: Topic Attention Networks for Neural Topic Modeling

Contact Info

Product

Resources

About