Analysis of Network Clustering Algorithms and Cluster Quality Metrics at Scale

Abstract

OverviewNotions of community quality underlie the clustering of networks. While studies surrounding network clustering are increasingly common, a precise understanding of the realtionship between different cluster quality metrics is unknown. In this paper, we examine the relationship between stand-alone cluster quality metrics and information recovery metrics through a rigorous analysis of four widely-used network clustering algorithms—Louvain, Infomap, label propagation, and smart local moving. We consider the stand-alone quality metrics of modularity, conductance, and coverage, and we consider the information recovery metrics of adjusted Rand score, normalized mutual information, and a variant of normalized mutual information used in previous work. Our study includes both synthetic graphs and empirical data sets of sizes varying from 1,000 to 1,000,000 nodes.Cluster Quality MetricsWe find significant differences among the results of the different cluster quality metrics. For example, clustering algorithms can return a value of 0.4 out of 1 on modularity but score 0 out of 1 on information recovery. We find conductance, though imperfect, to be the stand-alone quality metric that best indicates performance on the information recovery metrics. Additionally, our study shows that the variant of normalized mutual information used in previous work cannot be assumed to differ only slightly from traditional normalized mutual information.Network Clustering AlgorithmsSmart local moving is the overall best performing algorithm in our study, but discrepancies between cluster evaluation metrics prevent us from declaring it an absolutely superior algorithm. Interestingly, Louvain performed better than Infomap in nearly all the tests in our study, contradicting the results of previous work in which Infomap was superior to Louvain. We find that although label propagation performs poorly when clusters are less clearly defined, it scales efficiently and accurately to large graphs with well-defined clusters.

I/O-efficient Generation of Massive Graphs Following the LFR Benchmark

“…10. Similar to the results in [11], we also observe that the performance of clus- tering algorithms drops significantly as the graph's size grows. This might be due to less clearly defined community structures since the parameters are scaled, and also due to limits of current clustering algorithms.…”

“…For disjoint clusters we also compare it with the implementation that is part of NetworKit [35]. Using NetworKit, we evaluate the results of Infomap [32], Louvain [7] and OSLOM [23], three stateof-the-art clustering algorithms [8,11,14], and compare them using the adjusted rand measure [19] and NMI [12]. Further, we examine the average local clustering coefficient, a measure for the percentage of closed triangles which shows the presence of locally denser areas as expected in communities [20].…”

“…Synthetic benchmarks are commonly used for the evaluation of clustering algorithms since for most real-world networks it is unknown which clusters they contain and which of them are actually detectable through structure [3,14]. In the last years, the LFR benchmark [21,22] has become a standard benchmark for such experimental studies, both for disjoint and for overlapping communities [11].…”

Semi-automatic generation of a corpus of Wikipedia articles on science and technology

“…Infomap is known to be one of the best algorithms for this purpose Fortunato, 2009). Nevertheless, as stated in Emmons et al (2016), Infomap can fail to identify large communities in very large networks, as in our case. On the other hand, Infomap may also fail to generate accurate communities if there are too many small ones, as also occurs in our case.…”

“…Infomap clusters tightly interconnected nodes into modules (two-level clustering) or the optimal number of nested modules (multi-level clustering). It is known to be a fast method that outperforms other algorithms for community detection while providing a high modularity index, which is very suitable for large graphs (Emmons et al, 2016).…”