Dimensionality Reduction and Topic Modeling: From Latent Semantic Indexing to Latent Dirichlet Allocation and Beyond

Building sentiment lexicons based on recommending services for the Polish language

“…This is based on the assumptions that the importance of a word is proportional to the number of occurrences of this word in a document and inversely proportional to the number of documents in which the word occurred. The TF-IDF method can be used to find keywords from the text, but it fails to find a connection between semantically convergent documents that utilize different vocabularies, and more-complex methods need to be applied (such as Topic Modeling [7]). …”

“…Moreover, topic modeling can be used to reduce the representation of documents (e.g., using topics as features describing the text instead of all words). There are two main branches of Topic Modeling [7] -algorithms based on Singular Value Decomposition (such as Latent Semantic Indexing [13]) and those based on probabilistic generative processes [5] (such as Latent Dirichlet Allocation [3,4]). …”

Hybrid linear matrix factorization for topic-coherent terms clustering

“…The methods can group documents and words but cannot construct a knowledge structure as a topic model to allow the overlapping of documents and words in different topics. In Crain et al (2012), it combines Latent Semantic Indexing (LSI) and LDA for topic modeling. Its idea is to discuss the possibility of using these methods for large collection of text.…”

Feedback recurrent neural network-based embedded vector and its application in topic model

“…Statistical semantic models, such as latent semantic analysis (LSA) [3], probabilistic latent semantic analysis (PLSA) [4], and latent Dirichlet allocation (LDA) [5], are power tools for mining underlying topics in a document collection. PLSA is the probabilistic version of LSA, and both of them compute the similarities between documents with co-occurrences of terms, such as TF-IDF [6], but they ignore the order relationships between terms.…”

“…With these term frequency vectors, researchers could compute the similarities between documents and thus discover underlying topics in such a document collection [2]. Topic models can be classified into statistical semantic models [3][4][5][6][7][8][9][10] and embedded vector models [11][12][13]. While capturing the semantics of documents, statistical semantic model computes the similarities between documents with co-occurrence matrix of terms, and embedded vector model uses neighbor(s) to represent the meaning of a target term; however, both of them cannot describe the term orders in a document.…”