Improved mutual information measure for clustering, classification, and community detection

Abstract

The information theoretic quantity known as mutual information finds wide use in classification and community detection analyses to compare two classifications of the same set of objects into groups. In the context of classification algorithms, for instance, it is often used to compare discovered classes to known ground truth and hence to quantify algorithm performance. Here we argue that the standard mutual information, as commonly defined, omits a crucial term which can become large under real-world conditions, producing results that can be substantially in error. We demonstrate how to correct this error and define a mutual information that works in all cases. We discuss practical implementation of the new measure and give some example applications.

Significance-based multi-scale method for network community detection and its application in disease-gene prediction

“…Similarly, it may result in artificially large values, even when two non-random partitions are compared if these have a large number of groups. To counter balance for such bias, several metrics alternative to the NMI were introduced [59][60][61]. It seems that Significance tends to favor the detection of smallscale structures, potentially returning partitions with more communities (i.e.…”

Characterizing and Comparing External Measures for the Assessment of Cluster Analysis and Community Detection

“…For instance, Meilȃ [9] proves that the rescaling performed by some measures for normalization purposes, such as N M I, have the effect of breaking the Convex Additivity property. Moreover, Newman et al [30], like others [5], [10], [29], show that N M I tend to favor partitions with more clusters when compared with a reference partition (cf. no k-invariance), and that this behavior can be smoothed by correcting NMI for chance.…”

“…In this category, the most frequent property is probably kinvariance. Certain measures such as the Normalized Mutual Information tend to favor partitions depending on the number of clusters they contain when compared with a reference partition [5], a bias that a number of authors want to avoid [3], [26]- [30]. For example, suppose that one compares a ground truth partition to two estimated partitions differing only in their number of clusters.…”

“…Two variants mainly appear in the literature: splitting into equal [4], [9] vs. unequal parts [4], [9], [10], [29]. There is also a specific case of the first variant, consisting in splitting a cluster into only singleton clusters [15], [30], [42]. This transformation category is used in the literature to test several distinct properties.…”

“…Constant Baseline [3], [5], [10], [13], [20] ARI [1], AMI [26], rNMI [27], RMI [30], FNMI [29], cNMI [38] • Fragmenting every cluster [6] • Random shuffling [7], [13], [23], [25], [26], [28], [29] Baseline-Minimum Distinction [24], [25] ARI [1], SMI [13] • Random shuffling [25] Sensitivity to Partition Characteristics k-invariance [3], [26]- [30] ARI [1], VI [9] • Random shuffling [25]- [29] • Splitting into singleton clusters [30] • Swap with single cluster [7] n-invariance [3], [9], [17] VI [9], FMI [36], NMI [2], ARI [1] • None (proof)…”