Neural Topic Modeling by Incorporating Document Relationship Graph

Zhou, Deyu; Hu, Xuemeng; Wang, Rui

doi:10.18653/v1/2020.emnlp-main.310

Cited by 12 publications

(4 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Other methods such as have no code. We only include methods that leverage language models to do an apple-to-apple comparison and exclude methods using graph neural network (Zhou et al, 2020) or reinforcement learning (Costello and Reformat, 2023b).…”

Section: Compared Methods Selectionmentioning

confidence: 99%

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport

Xu¹,

Jiang²,

Rao³

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

Language model based methods are powerful techniques for text classification. However, the models have several shortcomings. (1) It is difficult to integrate human knowledge such as keywords. (2) It needs a lot of resources to train the models.(3) It relied on large text data to pretrain. In this paper, we propose Semi-Supervised vMF Neural Topic Modeling (S2vNTM) to overcome these difficulties. S2vNTM takes a few seed keywords as input for topics. S2vNTM leverages the pattern of keywords to identify potential topics, as well as optimize the quality of topics' keywords sets. Across a variety of datasets, S2vNTM outperforms existing semi-supervised topic modeling methods in classification accuracy with limited keywords provided. S2vNTM is at least twice as fast as baselines.

show abstract

Section: Compared Methods Selectionmentioning

confidence: 99%

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport

Xu¹,

Jiang²,

Rao³

et al. 2023

Findings of the Association for Computational Linguistics: ACL 2023

View full text Add to dashboard Cite

show abstract

“…A biterm refers to an unordered word pair that co-occurred in the same document, originally from Yan et al (2013). Similarly, Yang et al (2020); Zhou et al (2020) use a bipartite graph of documents and words, connected by word occurrences or TF-IDF values. Wang et al (2021b) use word co-occurrence and semantic correlation graphs.…”

Section: Ntms With Graph Neural Networkmentioning

confidence: 99%

A survey on neural topic models: methods, applications, and challenges

Wu,

Nguyen,

Luu

2024

Artif Intell Rev

View full text Add to dashboard Cite

Topic models have been prevalent for decades to discover latent topics and infer topic proportions of documents in an unsupervised fashion. They have been widely used in various applications like text analysis and context recommendation. Recently, the rise of neural networks has facilitated the emergence of a new research field—neural topic models (NTMs). Different from conventional topic models, NTMs directly optimize parameters without requiring model-specific derivations. This endows NTMs with better scalability and flexibility, resulting in significant research attention and plentiful new methods and applications. In this paper, we present a comprehensive survey on neural topic models concerning methods, applications, and challenges. Specifically, we systematically organize current NTM methods according to their network structures and introduce the NTMs for various scenarios like short texts and cross-lingual documents. We also discuss a wide range of popular applications built on NTMs. Finally, we highlight the challenges confronted by NTMs to inspire future research.

show abstract

“…GraphBTM [Zhu et al, 2018], based on the same assumption than BTM, represents biterms as a graph and uses a Graph Neural Network (GCN) [Kipf and Welling, 2017] to extract topic mixtures from them. Zhou et al [2020] proposes the Graph Topic Model (GTM), which extracts topic mixtures by processing a word-document graph with a GNN, to further improve topic coherence. Note that the word-document graph is distinct from an explicit document-document graph, as it solely computed from word occurrences in the documents.…”

Section: Neural Topic Modelsmentioning

confidence: 99%

Anchor Prediction: A Topic Modeling Approach

Dupuy,

Guille,

Jacques

2022

Preprint

View full text Add to dashboard Cite

Networks of documents connected by hyperlinks, such as Wikipedia, are ubiquitous. Hyperlinks are inserted by the authors to enrich the text and facilitate the navigation through the network. However, authors tend to insert only a fraction of the relevant hyperlinks, mainly because this is a time consuming task. In this paper we address an annotation, which we refer to as anchor prediction. Even though it is conceptually close to link prediction or entity linking, it is a different task that require developing a specific method to solve it. Given a source document and a target document, this task consists in automatically identifying anchors in the source document, i.e words or terms that should carry a hyperlink pointing towards the target document. We propose a contextualized relational topic model, CRTM, that models directed links between documents as a function of the local context of the anchor in the source document and the whole content of the target document. The model can be used to predict anchors in a source document, given the target document, without relying on a dictionary of previously seen mention or title, nor any external knowledge graph. Authors can benefit from CRTM, by letting it automatically suggest hyperlinks, given a new document and the set of target document to connect to. It can also benefit to readers, by dynamically inserting hyperlinks between the documents they're reading. Experiments conducted on several Wikipedia corpora (in English, Italian and German) highlight the practical usefulness of anchor prediction and demonstrate the relevancy of our approach.

show abstract

Neural Topic Modeling by Incorporating Document Relationship Graph

Cited by 12 publications

References 18 publications

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport

vONTSS: vMF based semi-supervised neural topic modeling with optimal transport

A survey on neural topic models: methods, applications, and challenges

Anchor Prediction: A Topic Modeling Approach

Contact Info

Product

Resources

About