A fast implementation of MLR-MCL algorithm on multi-core processors

Niu, Qingpeng; Lai, Pai-Wei; Faisal, Sutan; Parthasarathy, S.; Sadayappan, P.

doi:10.1109/hipc.2014.7116888

Cited by 13 publications

(10 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The authors of CODICIL displayed in the original paper [9] that when clustering a CODICIL sampled graph, they achieved a higher average F-score compared to methods such as LDA, PCL-DC, and K-means. In the original paper, Multi-Level Regularized Markov Clustering (MLR-MCL) [34] was used to cluster the generated graph. In this paper, regular Markov Clustering (MCL) [35] is used.…”

Section: Creating New Edges Based On Attributed Similarities Between mentioning

confidence: 99%

Interpretable Probabilistic Divisive Clustering of Large Node-Attributed Networks

Kaati

Ruul²

2017

2017 European Intelligence and Security Informatics Conference (EISIC)

View full text Add to dashboard Cite

Clustering of social networks, known as community detection is a fundamental part of social network analysis. A community (also known as a module or cluster) is a set of nodes grouped together according to some characteristic. Traditionally, a community has been thought of as a set of nodes that are more densely connected with each other than the rest of the network. Introducing node attributes to a social network, allows for two possible sources of information when clustering the network: The network structure, and the attributes describing the nodes. Traditional community detection methods supporting both these sources of information tend to be computationally complex and the resulting clusters are difficult to interpret in the sense of what characteristic they were grouped on. We present two methods (probabilistic divisive clustering and top-sampled community search) built on-top of already existing community detection methods (CESNA and FocusCO). Both of our methods aim to detect communities with a specified attribute association, yielding interpretable results in a feasible amount of time. The community detection algorithms our methods are built upon, are applied to different datasets in order to examine the runtime performance. We also display how our proposed methods can be used to detect communities formed around topics of interest and how a network can iteratively be clustered in order to detect subcommunities with a specified attribute association. In conjunction with research about psychological profiling, we believe that our proposed methods could be used to detect communities of people having similar psychological profile in online social networks.

show abstract

Section: Creating New Edges Based On Attributed Similarities Between mentioning

confidence: 99%

Interpretable Probabilistic Divisive Clustering of Large Node-Attributed Networks

Kaati

Ruul²

2017

2017 European Intelligence and Security Informatics Conference (EISIC)

View full text Add to dashboard Cite

show abstract

“…We consider only the first iteration of MCL, squaring the original adjacency matrix, as a representative example of the iteration. There have been several proposed variants of MCL, including (multi-level) regularized MCL [Satuluri and Parthasarathy 2009;Niu et al 2014], that perform slightly different SpGEMMs. We believe that the present experiments can help inform algorithmic choices for parallelizing any clustering algorithm applied to scale-free graphs that uses SpGEMM as its computational workhorse.…”

Section: Application: Markov Clusteringmentioning

confidence: 99%

“…Of the 11 non-synthetic data sets considered by Niu et al [2014], we present results for the 7 matrices that are publicly available and whose fine-grained hypergraphs fit in memory on our machine. The matrices dblp, enron, and facebook are social network matrices, roadnetca is a graph representing roads and intersections, and dip, wiphi, and biogrid11 are protein-protein interaction matrices.…”

Section: Application: Markov Clusteringmentioning

confidence: 99%

Hypergraph Partitioning for Sparse Matrix-Matrix Multiplication

Ballard

Druinsky

Knight

et al. 2016

ACM Trans. Parallel Comput.

View full text Add to dashboard Cite

We propose a fine-grained hypergraph model for sparse matrix-matrix multiplication (SpGEMM), a key computational kernel in scientific computing and data analysis whose performance is often communication bound. This model correctly describes both the interprocessor communication volume along a critical path in a parallel computation and also the volume of data moving through the memory hierarchy in a sequential computation. We show that identifying a communication-optimal algorithm for particular input matrices is equivalent to solving a hypergraph partitioning problem. Our approach is sparsity dependent, meaning that we seek the best algorithm for the given input matrices.In addition to our (3D) fine-grained model, we also propose coarse-grained 1D and 2D models that correspond to simpler SpGEMM algorithms. We explore the relations between our models theoretically, and we study their performance experimentally in the context of three applications that use SpGEMM as a key computation. For each application, we find that at least one coarse-grained model is as communication efficient as the fine-grained model. We also observe that different applications have affinities for different algorithms.Our results demonstrate that hypergraphs are an accurate model for reasoning about the communication costs of SpGEMM as well as a practical tool for exploring the SpGEMM algorithm design space.

show abstract

“…The MCL algorithm has been parallelized for multi-core and many-core systems [2], [3], however the current scale of the biological networks can make it imperative to use distributed memory systems. In this direction, a recent study [4] proposed the HipMCL algorithm for fast clustering of large-scale networks on distributed memory systems.…”

Section: Introductionmentioning

confidence: 99%