Text topic modeling via representation learning non-negative matrix factorization with semantic similarity

Abstract

Topic models are instrumental in text mining, revealing discriminative and coherent latent topics. Fewer words in short texts lead to insufficient contextual information and produce a highly sparse document-word matrix. So traditional topic models struggle to effectively cluster short texts. Models incorporating global word co-occurrence introduce too much information when processing long texts, resulting in a decrease in convergence speed and poorer clustering accuracy. To overcome sparsity in short texts and the impact of word co-occurrence on long texts, we propose a representation learning non-negative matrix factorization with semantic similarity topic model for texts of varying lengths, named RL-NMF-SS. The proposed method incorporates word co-occurrence and text similarity as regularization constraints and adjusts the regularization parameters to improve the adaptability to different corpora. Meanwhile, factor matrices are initialized via representation learning (RL) to bolster clustering robustness and model convergence. Extensive experiments on real-world corpora of varying text lengths, experimental results demonstrate RL-NMF-SS's superior performance in topic coherence and clustering accuracy, and RL-based initialization strategies exhibit excellent convergence.