An empirical study of the maximal and total information coefficients and leading measures of dependence

Abstract

In exploratory data analysis, we are often interested in identifying promising pairwise associations for further analysis while filtering out weaker ones. This can be accomplished by computing a measure of dependence on all variable pairs and examining the highest-scoring pairs, provided the measure of dependence used assigns similar scores to equally noisy relationships of different types. This property, called equitability and previously formalized, can be used to assess measures of dependence along with the power of their corresponding independence tests and their runtime. Here we present an empirical evaluation of the equitability, power against independence, and runtime of several leading measures of dependence. These include the two recently introduced and simultaneously computable statistics MIC e , whose goal is equitability, and TIC e , whose goal is power against independence. Regarding equitability, our analysis finds that MIC e is the most equitable method on functional relationships in most of the settings we considered. Regarding power against independence, we find that TIC e and Heller and Gorfine's S DDP share state-of-the-art performance, with several other methods achieving excellent power as well. Our analyses also show evidence for a trade-off between power against independence and equitability consistent with recent theoretical work. Our results suggest that a fast and useful strategy for achieving a combination of power against independence and equitability is to filter relationships by TIC e and then to rank the remaining ones using MIC e. We confirm our findings on a set of data collected by the World Health Organization.

A practical tool for maximal information coefficient analysis

“…2D and A1), is a result of the core approximation algorithm EQUICHARCLUMP, which speeds up the computation of MIC e [ 5 , 18 ]. The EQUICHARCLUMP parameter c controls the coarseness of the discretization in the grid search phase; by default, it is set to 5, providing good performance in most settings [ 10 ].…”

“…Two parameters control the estimation of the null distribution of TIC e : the parameter B controlling the maximal-allowed grid resolution and the number of permutations R . In the current implementation, B was set to the default value 9, which guarantees good performance in terms of statistical power against independence in most situations [ 10 ]. However, different values of B can be chosen; for example, B = 4 for less complex alternative hypothesis, B = 12 for more complex associations [ 10 ].…”

“…In the current implementation, B was set to the default value 9, which guarantees good performance in terms of statistical power against independence in most situations [ 10 ]. However, different values of B can be chosen; for example, B = 4 for less complex alternative hypothesis, B = 12 for more complex associations [ 10 ]. With regard to the number of permutations, instead, the results obtained on the synthetic datasets (see Additional File 2, Figs.…”

“…Recently, a family of measures based on the concept of mutual information has been proposed, and one of the most popular (and debated) members of this family, the maximal information coefficient (MIC), has been shown to have good equitability [ 1 ]. Unfortunately, MIC does not have state-of-the-art power [ 9 , 10 ], and its heuristic estimator, APPROX-MIC, is computationally demanding [ 5 ]. These two drawbacks have severely hampered the application of MIC to large datasets.…”

A visual analytics system for multi-model comparison on clinical data predictions

“…The recently developed measures, such as the mutual information (MI) [9], maximal information coefficient (MIC e ) [38], and total information coefficient (TIC e ) [40] fulfill the above requirements. Among these measures, TIC e is known for the best measure for various datasets [39,42]. We also tested these three measures on our dataset and TIC e produced the most reasonable conclusions.…”