Self-Supervised Neural Topic Modeling

Bahrainian, Seyed Ali; Jäggi, Martin; Eickhoff, Carsten

doi:10.18653/v1/2021.findings-emnlp.284

Cited by 4 publications

(1 citation statement)

References 7 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Bahrainian S A et al [21] proposed a new light-weight Self-Supervised Neural Topic Model (SNTM) that learns a rich context by learning a topic representation jointly from three co-occurring words and a document that the triplet originates from. Jin Y et al [22] proposed a variational autoencoder (VAE) NTM model that jointly reconstructs the sentence and document word counts using combinations of bag-of-words (BoW) topical embeddings and pre-trained semantic embeddings.…”

Section: Neural Topic Modelingmentioning

confidence: 99%

Neural Topic Modeling with Deep Mutual Information Estimation

Xu¹,

Lu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

The emerging neural topic models make topic modeling more easily adaptable and extendable in unsupervised text mining. However, the existing neural topic models is difficult to retain representative information of the documents within the learnt topic representation. In this paper, we propose a neural topic model which incorporates deep mutual information estimation, i.e., Neural Topic Modeling with Deep Mutual Information Estimation(NTM-DMIE). NTM-DMIE is a neural network method for topic learning which maximizes the mutual information between the input documents and their latent topic representation. To learn robust topic representation, we incorporate the discriminator to discriminate negative examples and positive examples via adversarial learning. Moreover, we use both global and local mutual information to preserve the rich information of the input documents in the topic representation. We evaluate NTM-DMIE on several metrics, including accuracy of text clustering, with topic representation, topic uniqueness and topic coherence. Compared to the existing methods, the experimental results show that NTM-DMIE can outperform in all the metrics on the four datasets.

show abstract

Section: Neural Topic Modelingmentioning

confidence: 99%

Neural Topic Modeling with Deep Mutual Information Estimation

Xu¹,

Lu²,

Li³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

ANTM: Aligned Neural Topic Models for Exploring Evolving Topics

Rahimi,

Naacke,

Constantin

et al. 2024

Lecture Notes in Computer Science

View full text Add to dashboard Cite

A self-supervised seed-driven approach to topic modelling and clustering

Ravenda,

Bahrainian,

Raballo

et al. 2024

J Intell Inf Syst

View full text Add to dashboard Cite

Topic models are useful tools for extracting the most salient themes within a collection of documents, grouping them to construct clusters representative of each specific topic. These clusters summarize and represent the semantic contents of the documents for better document interpretation. In this work, we present a light approach able to learn topic representations in a Self-Supervised fashion. More specifically, we propose a lightweight and scalable architecture using a seed-word driven approach to simultaneously co-learn a representation from a document and its corresponding word embeddings. The results obtained on a variety of datasets of different sizes and natures show that our model is capable of extracting meaningful topics. Furthermore, our experiments on five benchmark datasets illustrate that our model outperforms both traditional and neural topic modelling baseline models in terms of different coherence and clustering accuracy measures.

show abstract

Self-Supervised Neural Topic Modeling

Cited by 4 publications

References 7 publications

Neural Topic Modeling with Deep Mutual Information Estimation

Neural Topic Modeling with Deep Mutual Information Estimation

ANTM: Aligned Neural Topic Models for Exploring Evolving Topics

A self-supervised seed-driven approach to topic modelling and clustering

Contact Info

Product

Resources

About