GDTM: Graph-based Dynamic Topic Models

Abstract

Dynamic Topic Modeling (DTM) is the ultimate solution for extracting topics from short texts generated in Online Social Networks (OSNs) like Twitter. It requires to be scalable and to be able to account for sparsity and dynamicity of short texts. Current solutions combine probabilistic mixture models like Dirichlet Multinomial or Pitman-Yor Process with approximate inference approaches like Gibbs Sampling and Stochastic Variational Inference to, respectively, account for dynamicity and scalability of DTM. However, these methods basically rely on weak probabilistic language models, which do not account for sparsity in short texts. In addition, their inference is based on iterative optimizations, which have scalability issues when it comes to DTM. We present GDTM, a single-pass graph-based DTM algorithm, to solve the problem. GDTM combines a context-rich and incremental feature representation method with graph partitioning to address scalability and dynamicity and uses a rich language model to account for sparsity. We run multiple experiments over a large-scale Twitter dataset to analyze the accuracy and scalability of GDTM and compare the results with four state-of-the-art models. In result, GDTM outperforms the best model by $$11\%$$ 11 % on accuracy and performs by an order of magnitude faster while creating four times better topic quality over standard evaluation metrics.

Understanding Document Thematic Structure

“…Despite its successes, traditional topic modeling algorithms like LDA cannot efficiently model short text like posts on Twitter [12], [56]- [58]. LDA is limited when modeling documents with sparse matrix representation and therefore it is inefficient when modeling short text [57], [59].…”

“…LDA is limited when modeling documents with sparse matrix representation and therefore it is inefficient when modeling short text [57], [59]. Likewise, traditional LDA does not model topic correlation [38], [60], [61] and also, traditional topic modeling algorithms do not consider the evolution of topics over time [58], [62], [63].…”