EgoLP: Fast and Distributed Community Detection in Billion-Node Social Networks

Buzun, Nazar; Коршунов, Антон; Avanesov, Valeriy; Filonenko, Ilya; Kozlov, Ilya; Турдаков, Денис; Kim, Hangkyu

doi:10.1109/icdmw.2014.158

Cited by 25 publications

(26 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Coscia et al [16] improved over their clustering method by proposing to use a more sophisticated label propagation ego-net partitioning technique. Several authors have since built on such a line of work to improve the scalability and accuracy of ego-net based clustering [10,19,20,33,33], while others have designed ego-net analysis methods that tackle user metadata on top of the ego-net connectivity [31,32,49].…”

Section: Ego-net Analysismentioning

confidence: 99%

Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts

Epasto

Perozzi

2019

The World Wide Web Conference

104

109

View full text Add to dashboard Cite

Recent interest in graph embedding methods has focused on learning a single representation for each node in the graph. But can nodes really be best described by a single vector representation? In this work, we propose a method for learning multiple representations of the nodes in a graph (e.g., the users of a social network). Based on a principled decomposition of the ego-network, each representation encodes the role of the node in a different local community in which the nodes participate. These representations allow for improved reconstruction of the nuanced relationships that occur in the graph -a phenomenon that we illustrate through state-of-the-art results on link prediction tasks on a variety of graphs, reducing the error by up to 90%. In addition, we show that these embeddings allow for effective visual analysis of the learned community structure.

show abstract

Section: Ego-net Analysismentioning

confidence: 99%

Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts

Epasto

Perozzi

2019

The World Wide Web Conference

104

109

View full text Add to dashboard Cite

show abstract

“…With the emergence of massive networks that cannot be handled in the main memory of a single computer, new clustering schemes have been proposed for advanced models of computation [9,49]. Since such algorithms typically use hierarchical input representations, quality results of small benchmarks may not be generalizable to larger instances.…”

Section: Introductionmentioning

confidence: 99%

I/O-Efficient Generation of Massive Graphs Following the LFR Benchmark

Hamann

Meyer

Penschuck

et al. 2018

ACM J. Exp. Algorithmics

View full text Add to dashboard Cite

LFR is a popular benchmark graph generator used to evaluate community detection algorithms. We present EM-LFR, the first external memory algorithm able to generate massive complex networks following the LFR benchmark. Its most expensive component is the generation of random graphs with prescribed degree sequences which can be divided into two steps: the graphs are first materialized deterministically using the Havel-Hakimi algorithm, and then randomized. Our main contributions are EM-HH and EM-ES, two I/Oefficient external memory algorithms for these two steps. We also propose EM-CM/ES, an alternative sampling scheme using the Configuration Model and rewiring steps to obtain a random simple graph. In an experimental evaluation we demonstrate their performance; our implementation is able to handle graphs with more than 37 billion edges on a single machine, is competitive with a massive parallel distributed algorithm, and is faster than a state-of-theart internal memory implementation even on instances fitting in main memory. EM-LFR's implementation is capable of generating large graph instances orders of magnitude faster than the original implementation. We give evidence that both implementations yield graphs with matching properties by applying clustering algorithms to generated instances. Similarly, we analyse the evolution of graph properties as EM-ES is executed on networks obtained with EM-CM/ES and find that the alternative approach can accelerate the sampling process. * This work was partially supported by the DFG under grants ME 2088/3-2, WA 654/22-2. Parts of this paper were published as [21].

show abstract

“…For disjoint clusters we also compare it with the implementation that is part of NetworKit [35]. Using NetworKit, we evaluate the results of Infomap [32], Louvain [7] and OSLOM [23], three stateof-the-art clustering algorithms [8,11,14], and compare them using the adjusted rand measure [19] and NMI [12]. Further, we examine the average local clustering coefficient, a measure for the percentage of closed triangles which shows the presence of locally denser areas as expected in communities [20].…”

Section: Qualitative Comparison Of Em-lfrmentioning

confidence: 99%

“…Further, the implementation is capable of producing graphs with more than 1 × 10 10 edges in 17 h. 9 Using the same time budget, the original implementation generates graphs more than two orders of magnitude smaller. 8 We consider such vertices unrealistically (simple) as they have only degree 1 and account for ≈84 % of nodes in the original graph.…”

Section: Em-esmentioning

confidence: 99%

See 1 more Smart Citation

I/O-efficient Generation of Massive Graphs Following the LFR Benchmark

Hamann

Meyer

Penschuck

et al. 2017

2017 Proceedings of the Ninteenth Workshop on Algorithm Engineering and Experiments (ALENEX)

View full text Add to dashboard Cite

LFR is a popular benchmark graph generator used to evaluate community detection algorithms. We present EM-LFR, the first external memory algorithm able to generate massive complex networks following the LFR benchmark. Its most expensive component is the generation of random graphs with prescribed degree sequences which can be divided into two steps: the graphs are first materialized deterministically using the Havel-Hakimi algorithm, and then randomized. Our main contributions are EM-HH and EM-ES, two I/O-efficient external memory algorithms for these two steps. In an experimental evaluation we demonstrate their performance: our implementation is able to handle graphs with more than 37 billion edges on a single machine, is competitive with a massive parallel distributed algorithm, and is faster than a state-of-the-art internal memory implementation even on instances fitting in main memory. EM-LFR's implementation is capable of generating large graph instances orders of magnitude faster than the original implementation. We give evidence that both implementations yield graphs with matching properties by applying clustering algorithms to generated instances.

show abstract

EgoLP: Fast and Distributed Community Detection in Billion-Node Social Networks

Cited by 25 publications

References 26 publications

Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts

Is a Single Embedding Enough? Learning Node Representations that Capture Multiple Social Contexts

I/O-Efficient Generation of Massive Graphs Following the LFR Benchmark

I/O-efficient Generation of Massive Graphs Following the LFR Benchmark

Contact Info

Product

Resources

About