Dirichlet Process Mixture Model for Document Clustering with Feature Partition

Huang, Ruizhang; Yu, Guan; Wang, Zhaojun; Zhang, Jun; Shi, Liangxing

doi:10.1109/tkde.2012.27

Cited by 53 publications

(28 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this part, we compare the performance of GSDMM with K-means [13], HAC [15], and DMAFP [11]. K-means and HAC are two popular similarity-based clustering models.…”

Section: Comparison Of Clustering Modelsmentioning

confidence: 99%

“…In the experimental study, we compared GSDMM with Kmeans [13], the Hierarchical Agglomerative Clustering (HAC) model [15], and DMAFP [11]. We did not choose other clustering methods like Gaussian Mixture Model [5], Affinity Propagation [8], and Spectral clustering [18], because they are not scalable with high-dimensional and large-volume data like texts.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

A dirichlet multinomial mixture model-based approach for short text clustering

Yin

Wang

2014

Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

462

325

View full text Add to dashboard Cite

Short text clustering has become an increasingly important task with the popularity of social media like Twitter, Google+, and Facebook. It is a challenging problem due to its sparse, high-dimensional, and large-volume characteristics. In this paper, we proposed a collapsed Gibbs Sampling algorithm for the Dirichlet Multinomial Mixture model for short text clustering (abbr. to GSDMM). We found that GS-DMM can infer the number of clusters automatically with a good balance between the completeness and homogeneity of the clustering results, and is fast to converge. GSDMM can also cope with the sparse and high-dimensional problem of short texts, and can obtain the representative words of each cluster. Our extensive experimental study shows that GSDMM can achieve significantly better performance than three other clustering models.

show abstract

“…In this part, we compare the performance of GSDMM with K-means [13], HAC [15], and DMAFP [11]. K-means and HAC are two popular similarity-based clustering models.…”

Section: Comparison Of Clustering Modelsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

A dirichlet multinomial mixture model-based approach for short text clustering

Yin

Wang

2014

Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining

462

325

View full text Add to dashboard Cite

show abstract

“…Yu et al [40] and Huang et al [13] propose a Dirichlet process mixture with feature selection model (DPMFS) and a Dirichlet process mixture with feature partition model (DPMFP) for normal document clustering, respectively. They compare DPMFP with four other clustering models: EM text classification (EM-TC) [25], K-means [16], LDA [3] and exponential-family approximation of the Dirichlet compound multinomial distribution (EDCM) [8]; they find that DPMFP performs best.…”

Section: User Clustering and Text Clusteringmentioning

confidence: 99%

Explainable User Clustering in Short Text Streams

Zhao

Liang

Zhang

et al. 2016

Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval

View full text Add to dashboard Cite

User clustering has been studied from different angles: behaviorbased, to identify similar browsing or search patterns, and contentbased, to identify shared interests. Once user clusters have been found, they can be used for recommendation and personalization. So far, content-based user clustering has mostly focused on static sets of relatively long documents. Given the dynamic nature of social media, there is a need to dynamically cluster users in the context of short text streams. User clustering in this setting is more challenging than in the case of long documents as it is difficult to capture the users' dynamic topic distributions in sparse data settings. To address this problem, we propose a dynamic user clustering topic model (or UCT for short). UCT adaptively tracks changes of each user's time-varying topic distribution based both on the short texts the user posts during a given time period and on the previously estimated distribution. To infer changes, we propose a Gibbs sampling algorithm where a set of word-pairs from each user is constructed for sampling. The clustering results are explainable and human-understandable, in contrast to many other clustering algorithms. For evaluation purposes, we work with a dataset consisting of users and tweets from each user. Experimental results demonstrate the effectiveness of our proposed clustering model compared to state-of-the-art baselines.

show abstract

“…However, such data encryptions render effective data utilization a very challenging task due to the basic reason that there could be a large amount of outsourced data files. Not only that, many a times, in Cloud Computing, data owners may share their complete outsourced data with a large number of users, who actually might want to retrieve only certain specific data files that they are interested in during a given session (Ming et al, 2011;Murugesan, 2011;Huang et al, 2013;Song et al, 2000;Wang et al, 2010;2012;Witten et al, 1999;Yu et al, 2013).…”

Section: Ajasmentioning

confidence: 99%

Efficient Ranked and Secure File Retrieval in Cloud Computing

Jeya¹

2014

American Journal of Applied Sciences

View full text Add to dashboard Cite

Cloud computing facilitates extremely scalable services that can be consumed over internet. An important aspect of the cloud services is that user data are stored remotely in unknown machines in which users do not posses or manage. Since the data's are stored remotely, we have to keep in mind that sensitive cloud data have to be encrypted before they are outsourced to the commercial public cloud, which makes efficient data utilization service. Searchable encryption file retrieval technique allows users to securely search over encrypted data through search word. Ranking the files based on relevance scores greatly enhances system usability by making it possible relevance ranking instead of sending unwanted results and further ensures the file retrieval accuracy. In this study, we are developing an automated system for both named and unnamed documents based on the clustering algorithms. We implement the ranking and searching algorithm to retrieve top k files. We also provides the mapping and encryption algorithm to protect the information. The resulting design is able to provide efficient ranking which will reduce the search time drastically and reduce the communication overhead. The mapping and encryption algorithms protect document against an outside attackers and prevent an untrusted cloud data provider from learning data.

show abstract

Dirichlet Process Mixture Model for Document Clustering with Feature Partition

Cited by 53 publications

References 24 publications

A dirichlet multinomial mixture model-based approach for short text clustering

A dirichlet multinomial mixture model-based approach for short text clustering

Explainable User Clustering in Short Text Streams

Efficient Ranked and Secure File Retrieval in Cloud Computing

Contact Info

Product

Resources

About