Short Text Topic Modeling with Flexible Word Patterns

Wu, Xiaobao; Li, Chunping

doi:10.1109/ijcnn.2019.8852366

Cited by 14 publications

(10 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…To improve the performance of short text topic modeling, Biterm Topic Model (BTM) (Yan et al, 2013) and Dirichlet Multinomial Mixture (DMM) model (Nigam et al, 2000;Sadamitsu et al, 2007;Yin and Wang, 2014) are two basic short text probabilistic topic models which employ traditional Bayesian inference methods including Gibbs Sampling (Steyvers and Griffiths, 2007) and Variational Inference (Blei et al, 2017). Several extensions based on BTM and DMM are also proposed, such as Generalized Pólya Urn-DMM (GPUDMM) (Li et al, 2016) with word embeddings and Multiterm Topic Model (Wu and Li, 2019). Besides, Semantics-assisted Non-negative Matrix Factorization (SeaNMF) (Shi et al, 2018) was lately proposed as an NMF topic model incorporating word-context semantic correlations solved by a block coordinate descent algorithm.…”

Section: Related Workmentioning

confidence: 99%

Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder

Wu¹,

Li²,

Zhu³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

Topic models have been prevailing for many years on discovering latent semantics while modeling long documents. However, for short texts they generally suffer from data sparsity because of extremely limited word cooccurrences; thus tend to yield repetitive or trivial topics with low quality. In this paper, to address this issue, we propose a novel neural topic model in the framework of autoencoding with a new topic distribution quantization approach generating peakier distributions that are more appropriate for modeling short texts. Besides the encoding, to tackle this issue in terms of decoding, we further propose a novel negative sampling decoder learning from negative samples to avoid yielding repetitive topics. We observe that our model can highly improve short text topic modeling performance. Through extensive experiments on real-world datasets, we demonstrate our model can outperform both strong traditional and neural baselines under extreme data sparsity scenes, producing high-quality topics.

show abstract

Section: Related Workmentioning

confidence: 99%

Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder

Wu¹,

Li²,

Zhu³

et al. 2020

Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, in this model, some words fail to have semantic relevance and tend to reduce efficiency. Wu and Li (2019) established the Multi-term Topic Model (MTM), which extracts the length of variable and multiple correlative word patterns from short texts to infer the latent trending topics. This model overcomes the limitations of BTM, such as extracting many irrelevant and useless bi-terms.…”

Section: Global Word Co-occurrences Based Asttm Modelsmentioning

confidence: 99%

Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis

et al. 2022

View full text Add to dashboard Cite

Social media platforms such as (Twitter, Facebook, and Weibo) are being increasingly embraced by individuals, groups, and organizations as a valuable source of information. This social media generated information comes in the form of tweets or posts, and normally characterized as short text, huge, sparse, and low density. Since many real-world applications need semantic interpretation of such short texts, research in Short Text Topic Modeling (STTM) has recently gained a lot of interest to reveal unique and cohesive latent topics. This article examines the current state of the art in STTM algorithms. It presents a comprehensive survey and taxonomy of STTM algorithms for short text topic modelling. The article also includes a qualitative and quantitative study of the STTM algorithms, as well as analyses of the various strengths and drawbacks of STTM techniques. Moreover, a comparative analysis of the topic quality and performance of representative STTM models is presented. The performance evaluation is conducted on two real-world Twitter datasets: the Real-World Pandemic Twitter (RW-Pand-Twitter) dataset and Real-world Cyberbullying Twitter (RW-CB-Twitter) dataset in terms of several metrics such as topic coherence, purity, NMI, and accuracy. Finally, the open challenges and future research directions in this promising field are discussed to highlight the trends of research in STTM. The work presented in this paper is useful for researchers interested in learning state-of-the-art short text topic modelling and researchers focusing on developing new algorithms for short text topic modelling.

show abstract

“…Multiple machine learning based implementation of various topic modelling techniques have been extensively studied by many researchers [ 2,3,4,5,6]. Organizing, searching, and summarizing large volume of textual data are the major tasks involved in NLP and the topic modelling can be used to solve these problems to certain extent.…”

Section: Review Of Related Workmentioning

confidence: 99%

“…There is a demand for an automatic topic modeling system and text summarization, due to the enormous volume of text data available in present times and the limitation of human reading abilities. There are multiple techniques used to discover the topics from text, images, and video [1] Topic models have been extensively utilized in Topic Recognition and Tracking tasks, which helps to track, detect, and designate topics from a stream of documents or texts [4]. This machine learning technique is widely used in NLP applications to analyze the unstructured textual data and automatically discover the abstract topics within it.…”

Section: Introductionmentioning

confidence: 99%

An Integrated Clustering and BERT Framework for Improved Topic Modeling

George

Sumathy

2022

Preprint

View full text Add to dashboard Cite

Topic modelling is a machine learning technique that is extensively used in Natural Language Processing (NLP) applications to infer topics within unstructured textual data. Latent Dirichlet Allocation (LDA) is one of the most used topic modeling techniques that can automatically detect topics from a huge collection of text documents. However, the LDA-based topic models alone do not always provide promising results. Clustering is one of the effective unsupervised machine learning algorithms that are extensively used in applications including extracting information from unstructured textual data and topic modeling. A hybrid model of Bidirectional Encoder Representations from Transformers (BERT) and Latent Dirichlet Allocation (LDA) in topic modeling with clustering have been studied in detail. As the clustering algorithms are computationally complex, the complexity increases with the higher number of features, the PCA, t-SNE & UMAP-based dimensionality reduction methods are also performed. Finally, a unified clustering-based framework using BERT and LDA is proposed as part of this study for mining a set of meaningful topics from the massive text corpora. The experiments are conducted to demonstrate the effectiveness of the cluster-informed topic modeling framework using BERT and LDA by simulating user input on benchmark datasets. The experimental results show that clustering would help infer more coherent topics and hence this unified clustering and BERT-LDA based approach can be effectively utilized for building topic modeling applications.

show abstract

Short Text Topic Modeling with Flexible Word Patterns

Cited by 14 publications

References 19 publications

Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder

Short Text Topic Modeling with Topic Distribution Quantization and Negative Sampling Decoder

Short text topic modelling approaches in the context of big data: taxonomy, survey, and analysis

An Integrated Clustering and BERT Framework for Improved Topic Modeling

Contact Info

Product

Resources

About