Clustering memes in social media streams

Abstract

The problem of clustering content in social media has pervasive applications, including the identification of discussion topics, event detection, and content recommendation. Here we describe a streaming framework for online detection and clustering of memes in social media, specifically Twitter. A pre-clustering procedure, namely protomeme detection, first isolates atomic tokens of information carried by the tweets. Protomemes are thereafter aggregated, based on multiple similarity measures, to obtain memes as cohesive groups of tweets reflecting actual concepts or topics of discussion. The clustering algorithm takes into account various dimensions of the data and metadata, including natural language, the social network, and the patterns of information diffusion. As a result, our system can build clusters of semantically, structurally, and topically related tweets. The clustering process is based on a variant of Online K-means that incorporates a memory mechanism, used to "forget" old memes and replace them over time with the new ones. The evaluation of our framework is carried out by using a dataset of Twitter trending topics. Over a one-week period, we systematically determined whether our algorithm was able to recover the trending hashtags. We show that the proposed method outperforms baseline algorithms that only use content features, as well as a state-of-the-art event detection method that assumes full knowledge of the underlying follower network. We finally show that our online learning framework is flexible, due to its independence of the adopted clustering algorithm, and best suited to work in a streaming scenario.

Who creates Trends in Online Social Media: The Crowd or Opinion Leaders?

“…The authors also find that exogenous factors are more important than epidemic spreading in establishing hashtag popularity. JafariAsbagh et al () propose a streaming framework for detecting and clustering memes in online social networks. Bao et al (2013) investigate the cumulative effect of information diffusion on Weibo and argue that additional exposures do not improve the probability of retweets.…”

The thematic modelling of subtext

“…In previous work we have shown that it possible to provide a guide that leads to the creation of effective models, and this does provide some systematic structure to the creation of thematic models. However, more work is needed to explore whether models could be constructed in an automatic way, for example by using clustering techniques to derive coherent terms and concepts from social media streams [20].…”

Event identification in social networks

“…For example, Twitter allows users to utilize the "#" symbol, called hashtag, to mark keywords or topics in a Tweet; an image is usually associated with multiple labels which are characterized by different regions in the image; users are able to build connection with others (link information). Previous text analytics sources most often appear as <user, content> structure, while the text analytics in social media is able to derive data from various aspects, which include user, content, link, tag, timestamps and others [62,63,64].…”