Self-falsifiable hierarchical detection of overlapping communities on social networks

Abstract

No community detection algorithm can be optimal for all possible networks, thus it is important to identify whether the algorithm is suitable for a given network. We propose a multi-step algorithmic solution scheme for overlapping community detection based on an advanced label propagation process, which imitates the community formation process on social networks. Our algorithm is parameter-free and is able to reveal the hierarchical order of communities in the graph. The unique property of our solution scheme is self-falsifiability; an automatic quality check of the results is conducted after the detection, and the fitness of the algorithm for the specific network is reported. Extensive experiments show that our algorithm is self-consistent, reliable on networks of a wide range of size and different sorts, and is more robust than existing algorithms on both sparse and large-scale social networks. Results further suggest that our solution scheme may uncover features of networks' intrinsic community structures.Community detection is a central topic in network science. Pioneered by works represented by Palla et al [1], in recent years more and more studies focus on the detection of overlapping communities as opposed to exhaustive communities, a division also regarded as soft-partitioning versus hard-partitioning. Besides relying on the optimization of certain metrics, e.g. modularity [2], conductance [3], fitness [4], or aggregative metric based on link prediction algorithms [5] etc, as the extensively-studied traditional approach for detecting exhaustive communities, for overlapping communities, a lot of new tools are designed based on various ideas, including link communities [6, 7], clique percolation [8], seed set expansion [3,[9][10][11][12], label propagation [13][14][15][16], local spectral clustering method [17], and methods based on statistical inference, such as Infomap [18], the stochastic block model (SBM,[19,20]; in particular, methods adopting the belief propagation algorithm [21,22]), and other generative models [23,24].Despite the success of different solution schemes on various application fronts, some weak points of existing community detection algorithms could be pinned down in practice, which we believe might be problematic in certain cases (see appendix A for a detailed discussion). These weaknesses include: (1) many solution schemes are over-parameterized, and in some cases the tuning of parameters depends largely on unwarranted heuristics;(2) many scalable methods based on the seed set expansion process [2-4, 9, 25-28] may lack well-designed seeding strategies [10,11,29] and often rely on ad-hoc strategies; (3) some algorithms that claim to be local, as opposed to methods based on an optimization over the entire graph, in fact still optimize on the community level and thus do not guarantee complete locality; (4) the number of communities in the graph is often predetermined in certain algorithms, which might not be a good treatment, despite its claimed advantage [30] and the possible determina...

A Genetic Algorithm-Based Methodology for Analyzing the Characteristics of High-Operational-Capability Combat Networks

“…Here, as shown in Fig. 1, we first apply the community detection algorithm [33] to obtain a cartographic representation of the high-capability network to analyze its topological structure [34]. The cartographic representation, consisting of community modules, provides a simplified description of the HCN and thus can conveniently summarize information about the nodes and links.…”

Research on Overlapping Communities in Scientific Cooperation Network

“…The process of seed nodes spreading labels to other nodes is equivalent to the process of researchers following star scientists. The label propagation algorithm based on seed diffusion in reference [9] also adds the detection mechanism of overlapping communities. Based on this algorithm, this paper makes some improvements and forms an overlapping community detection algorithm suitable for scientific cooperation networks.…”

Detecting hierarchical organization of pervasive communities by modular decomposition of Markov chain

“…Hierarchical organization of communities is another essential aspect of real-world networks 5 , 7 , 15 – 23 . As noted by Simon 24 , the hierarchy in modular structure is a fundamental architecture of social and biological systems.…”

“…Turning now to pervasive communities, we realize that little has been discussed about their hierarchical organization so far. Given that the hierarchy and pervasiveness are both hallmarks of community structure of real-world networks, it may be surprising that few scholars have attempted to unify them, albeit several studies conducted to merge the hierarchy and overlapping of definite communities 5 , 7 , 23 , 25 . A difficulty lies in that, since pervasive structure no longer defines a single or a finite number of communities to which a given node belongs, the tracing strategy to identify parent–child relationship as above is no longer effective.…”