Comparative study of clustering techniques for short text documents

Rangrej, Aniket; Kulkarni, Siddhivinayak; Tendulkar, Ashish V.

doi:10.1145/1963192.1963249

Cited by 59 publications

(35 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…They compare DPMFP with four other clustering models: EM text classification (EM-TC) [25], K-means [16], LDA [3] and exponential-family approximation of the Dirichlet compound multinomial distribution (EDCM) [8]; they find that DPMFP performs best. In the context of short text documents, Rangrej et al [27] compare three clustering algorithms including Kmeans, Singular Value Decomposition and Affinity Propagation [9] on a small set of tweets and find that Affinity Propagation outperforms the other two, but the complexity of Affinity Propagation is quadratic in the number of documents. Tsur et al [33], Yin [38], and Yu et al [40] focus on the problem of online clustering of a stream of tweets.…”

Section: User Clustering and Text Clusteringmentioning

confidence: 99%

Explainable User Clustering in Short Text Streams

Zhao

Liang

Zhang

et al. 2016

Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

User clustering has been studied from different angles: behaviorbased, to identify similar browsing or search patterns, and contentbased, to identify shared interests. Once user clusters have been found, they can be used for recommendation and personalization. So far, content-based user clustering has mostly focused on static sets of relatively long documents. Given the dynamic nature of social media, there is a need to dynamically cluster users in the context of short text streams. User clustering in this setting is more challenging than in the case of long documents as it is difficult to capture the users' dynamic topic distributions in sparse data settings. To address this problem, we propose a dynamic user clustering topic model (or UCT for short). UCT adaptively tracks changes of each user's time-varying topic distribution based both on the short texts the user posts during a given time period and on the previously estimated distribution. To infer changes, we propose a Gibbs sampling algorithm where a set of word-pairs from each user is constructed for sampling. The clustering results are explainable and human-understandable, in contrast to many other clustering algorithms. For evaluation purposes, we work with a dataset consisting of users and tweets from each user. Experimental results demonstrate the effectiveness of our proposed clustering model compared to state-of-the-art baselines.

show abstract

Section: User Clustering and Text Clusteringmentioning

confidence: 99%

Explainable User Clustering in Short Text Streams

Zhao

Liang

Zhang

et al. 2016

Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

show abstract

“…He proposed and described a method to determine the most appropriate topic model fortweet clustering. Rangrej et al (2011) compared various document clustering techniques including k-means, SVD-based method and a graphbased approach and compared their performance on short text data collected from Twitter. Tweet Motif that clusters Twitter messages by frequent significant terms was presented by O'Connor et al (2010).…”

Section: Related Workmentioning

confidence: 99%

“…Since English is the most commonly used language in Twitter (Honey and Herring, 2009), the focus is on tweets written in English. Twitter provides a large quantity of short text in the form of tweets where each tweet represents a single document (Rangrej et al, 2011). Goyal (2011) stated that tweet similarity between two users is defined as "the cosine similarity between the documents formed by combining the tweets of a user into one".…”

Section: The Problemmentioning

confidence: 99%

Clustering Tweets Using Cellular Genetic Algorithm

Adel¹

2014

Journal of Computer Science

View full text Add to dashboard Cite

As the popularity of Twitter continues to increase rapidly, it is extremely necessary to analyze the huge amount of data that Twitter users generate. A popular method of tweet analysis is clustering. Because most tweets are textual, this study focuses on clustering tweets based on their textual content similarity. This study presents tweet clustering using cellular genetic algorithm cGA. The results obtained by cGA are compared with those obtained by generational genetic algorithm in terms of average fitness, average time required for execution and number of generations. Experimental results are tested with two sets: One of 1000 tweets and the second formed of 5000 tweets. The results show a nearly equal performance for both algorithms in terms of the average fitness of the solution. On the other hand, cGA shows a much faster performance than generational. These results demonstrate that cellular genetic algorithm outperforms generational genetic algorithm in tweet clustering.

show abstract

“…Therefore, TDM [14] has both advantages of EDM and CDM which are scale invariant while allowing discrimination collinear vectors. In accordance with performance of distance measure, TDM [14,15,16,17] is one of best distance measure for clustering text. Because of several reasons stated before, it is important to understand the behavior of TDM in distributed environment especially in DKM and DFCM.…”

Section: Introductionmentioning

confidence: 99%

Comparison of Distributed K-Means and Distributed Fuzzy C-Means Algorithms for Text Clustering

Agastya

Adji

Setiawan

2017

CST

View full text Add to dashboard Cite

Text clustering has been developed in distributed system due to increasing data. The popular algorithms like K-Means (KM) and Fuzzy CMeans (FCM) are combined with Map Reduce algorithm in Hadoop Environment to be distributable and parallelizable. The problem is performance comparison between Distributed KM (DKM) and Distributed FCM (DFCM) that uses Tanimoto Distance Measure (TDM) has not been studied yet. It is important because TDM's characteristics are scale invariant while allowing discrimination collinear vectors. This work compared the combination of TDM with DKM (DKM-T) and TDM with DFCM (DFCM-T) to acquire performance of both algorithms. The result shows that DFCM-T has better intra-cluster and inter-cluster densities than those of DKM-T. Moreover, DFCM-T has lower processing time than that of DKM-T when total nodes used are 4 and 8. DFCM-T and DKM-T can perform clustering of 1,400,000 text files in 16.18 and 9.74 minutes but the preprocessing times take hours to complete.

show abstract

Comparative study of clustering techniques for short text documents

Cited by 59 publications

References 4 publications

Explainable User Clustering in Short Text Streams

Explainable User Clustering in Short Text Streams

Clustering Tweets Using Cellular Genetic Algorithm

Comparison of Distributed K-Means and Distributed Fuzzy C-Means Algorithms for Text Clustering

Contact Info

Product

Resources

About