Accounting ngrams and multi-word terms can improve topic models

Abstract

The paper presents an empirical study of integrating ngrams and multi-word terms into topic models, while maintaining similarities between them and words based on their component structure. First, we adapt the PLSA-SIM algorithm to the more widespread LDA model and ngrams. Then we propose a novel algorithm LDA-ITER that allows the incorporation of the most suitable ngrams into topic models. The experiments of integrating ngrams and multiword terms conducted on five text collections in different languages and domains demonstrate a significant improvement in all the metrics under consideration.

Combining Thesaurus Knowledge and Probabilistic Topic Models

“…As it was found before [15,17], the addition of ngrams without accounting relations between their components considerably worsens the perplexity because of the vocabulary growth (for perplexity the less is the better) and practically does not change other automatic quality measures ( Table 2).…”

“…At the preprocessing step, documents were processed by morphological analyzers. Also, we extracted noun groups as described in [17]. As baselines, we use the unigram LDA topic model and LDA topic model with added 1000 ngrams with maximal NCvalue [21] extracted from the collection under analysis.…”

“…And we see the great improvement of the model quality: the kernel uniqueness considerably improves, perplexity decreases to levels comparable with the unigram model, topic coherence characteristics also improve for most collections ( Table 2:LDA-Sim+WNsynrel/hyp). We further use the WordNet-based similarity sets with n-grams having the same components as described in [17]. All measures significantly improve for all collections ( Table 2:LDA-Sim+WNsr/hyp+Ngrams).…”

Handling Collocations in Hierarchical Latent Tree Analysis for Topic Modeling

“…Only bigrams were considered in their study. Nokel and Loukachevitch [14] followed the collocation extraction approach. They modified the parameter estimation method LDA to such that bigrams and unigrams belong to the same topics more often.…”

Analysis of MWES in Hindi Text Using NLTK

“…More detailed knowledge and syntax related information is given in [13][14][15]. In [1] an empirical study of integrating n-grams and multi-word terms into topic models is presented by the authors while maintaining similarities between them and words based on their component structure with the help of LDA-ITER algorithm by which the most suitable n-grams and multiword terms are incorporated. In [4], the author thoroughly examined various types of MWE encountered in Hindi from machine translation viewpoint.…”