Defining and evaluating network communities based on ground-truth

Abstract

Nodes in real-world networks organize into densely linked communities where edges appear with high concentration among the members of the community. Identifying such communities of nodes has proven to be a challenging task mainly due to a plethora of definitions of a community, intractability of algorithms, issues with evaluation and the lack of a reliable gold-standard ground-truth. In this paper we study a set of 230 large real-world social, collaboration and information networks where nodes explicitly state their group memberships. For example, in social networks nodes explicitly join various interest based social groups. We use such groups to define a reliable and robust notion of ground-truth communities. We then propose a methodology which allows us to compare and quantitatively evaluate how different structural definitions of network communities correspond to ground-truth communities. We choose 13 commonly used structural definitions of network communities and examine their sensitivity, robustness and performance in identifying the ground-truth. We show that the 13 structural definitions are heavily correlated and naturally group into four classes. We find that two of these definitions, Conductance and Triad-participation-ratio, consistently give the best performance in identifying ground-truth communities. We also investigate a task of detecting communities given a single seed node. We extend the local spectral clustering algorithm into a heuristic parameter-free community detection method that easily scales to networks with more than hundred million nodes. The proposed method achieves 30% relative improvement over current local clustering methods.

Friend Grouping Algorithms for Online Social Networks: Preference, Bias, and Implications

“…This approach assumes users know and can identify real groups within their structural social networks. In a similar vein, a few studies started to collect the groundtruth data from different social networks by asking people to label their groups [22,28]. None of these studies, however, evaluate the effect of automating group detection for grouping friends by OSNs users.…”

“…Recent work evaluated these metrics by applying them to social, collaboration, and information networks where the nodes had explicit group memberships. Of the thirteen evaluated metrics, we chose the two with the best consistent reported performance in identifying ground-truth communities: Conductance and Triad-Participation Ratio [20,28].…”

“…-Triad-Participation Ratio (TPR): TPR metric is the fraction of members in a group that belong to a triad, a set of three connected nodes, inside the group. Unlike conductance, a higher TPR represents a tighter group [28]. We measured these metrics over predicted and desired groups to compare their values before and after users' modification.…”

Overlapping Communities for Identifying Misbehavior in Network Communications

“…However, there is no consensus on a single definition for a community and a variety of definitions have been used in the literature [13,19,28]. For network intrusion detection, Ding et al [9] defined a community as a group of source nodes that communicate with at least one common destination.…”

“…The framework allows us to incorporate different community detection algorithms for identifying anomalous nodes that belong to multiple communities. However, since legitimate nodes can also belong to several communities [28], applicationspecific filters can be used for discriminating the legitimate nodes from the antisocial nodes in the community overlaps, thus reducing the induced false positives.…”