Issues of Stability and Uniqueness of Stochastic Matrix Factorization

Derbanosov, R. Yu.; Irkhin, I. A.

doi:10.1134/s0965542520030082

Cited by 1 publication

(1 citation statement)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…One of the main problems of topic modeling is instability i. e. convergence to different solutions from different initializations. Mathematical origins of this issue were studied in [6], [9], [18], [5] where authors research the problem of uniqueness of Nonnegative Matrix Factorization (NMF). Another approach to the problem is customization of basic algorithms to achieve better stability.…”

Section: Introductionmentioning

confidence: 99%

Stability of Topic Modeling via Modality Regularization

Derbanosov¹,

Bakhanova²

2020

Computational Linguistics and Intellectual Technologies

View full text Add to dashboard Cite

Probabilistic topic modeling is a tool for statistical text analysis that can give us information about the inner structure of a large corpus of documents. The most popular models—Probabilistic Latent Semantic Analysis and Latent Dirichlet Allocation—produce topics in a form of discrete distributions over the set of all words of the corpus. They build topics using an iterative algorithm that starts from some random initialization and optimizes a loss function. One of the main problems of topic modeling is sensitivity to random initialization that means producing significantly different solutions from different initial points. Several studies showed that side information about documents may improve the overall quality of a topic model. In this paper, we consider the use of additional information in the context of the stability problem. We represent auxiliary information as an additional modality and use BigARTM library in order to perform experiments on several text collections. We show that using side information as an additional modality improves topics stability without significant quality loss of the model.

show abstract

Section: Introductionmentioning

confidence: 99%