Improving LDA topic models for microblogs via tweet pooling and automatic labeling

Abstract

Twitter, or the world of 140 characters poses serious challenges to the efficacy of topic models on short, messy text. While topic models such as Latent Dirichlet Allocation (LDA) have a long history of successful application to news articles and academic abstracts, they are often less coherent when applied to microblog content like Twitter. In this paper, we investigate methods to improve topics learned from Twitter content without modifying the basic machinery of LDA; we achieve this through various pooling schemes that aggregate tweets in a data preprocessing step for LDA. We empirically establish that a novel method of tweet pooling by hashtags leads to a vast improvement in a variety of measures for topic coherence across three diverse Twitter datasets in comparison to an unmodified LDA baseline and a variety of pooling schemes. An additional contribution of automatic hashtag labeling further improves on the hashtag pooling results for a subset of metrics. Overall, these two novel schemes lead to significantly improved LDA topic models on Twitter content.

Exploring Time-Sensitive Variational Bayesian Inference LDA for Social Media Data

“…In the M step of EM, φ D×N ×K is used to update the variational Dirichlet prior λ K×N of β. The parameters' optimisation formulas in the EM algorithm are shown in Equation (2). Finally, β and θ can be obtained when the lower bound converges.…”

“…Assigning all words in a tweet to a single topic could increase word overlaps and thus result in mixed topics. 2) Multiple posts are combined into a virtual document [2,11], also known as the pooling strategy (e.g. tweets from a single user are combined into a single document [2]).…”

“…2) Multiple posts are combined into a virtual document [2,11], also known as the pooling strategy (e.g. tweets from a single user are combined into a single document [2]). The pooling method can increase the number and occurrence of words, which makes it easier to apply a topic modelling approach.…”

“…Yet despite social media's ubiquity, scholars still wrestle with the appropriate tools for best capturing the topics of discussion conveyed over these platforms [1][2][3]. To this end, researchers have employed various topic modelling approaches [1,[4][5][6][7][8], e.g.…”

“…Latent Dirichlet Allocation (LDA), but these efforts have proved challenging, as models applied to social media data can produce topics that are mixed and lack coherence, and are generally difficult to interpret [1]. To deal with the short nature of social media posts, LDA enhancement methods such as single topic assignment [1,9,10] and document pooling [2,11] have been proposed to improve the coherence of the generated topics within the sampling-based topic modelling approaches. However, another LDA implementation, the Variational Bayesian (VB)-based topic modelling approach, has not been well studied on social media posts.…”