Asymptotic Distributions of Empirical Interaction Information

Abstract

Interaction Information is one of the most promising interaction strength measures with many desirable properties. However, its use for interaction detection was hindered by the fact that apart from the simple case of overall independence, asymptotic distribution of its estimate has not been known. In the paper we provide asymptotic distributions of its empirical versions which are needed for formal testing of interactions. We prove that for three-dimensional nominal vector normalized empirical interaction information converges to the normal law unless the distribution coincides with its Kirkwood approximation. In the opposite case the convergence is to the distribution of weighted centred chi square random variables. This case is of special importance as it roughly corresponds to interaction information being zero and the asymptotic distribution can be used for construction of formal tests for interaction detection. The result generalizes result in Han (Inf Control 46(1):26-45 1980) for the case when all coordinate random variables are independent. The derivation relies on studying structure of covariance matrix of asymptotic distribution and its eigenvalues. For the case of 3 × 3 × 2 contingency table corresponding to study of two interacting Single Nucleotide Polymorphisms (SNPs) for prediction of binary outcome, we provide complete description of the asymptotic law and construct approximate critical regions for testing of interactions when two SNPs are possibly dependent. We show in numerical experiments that the test based on the derived asymptotic distribution is easy to implement and yields actual significance levels consistently closer to the nominal ones than the test based on chi square reference distribution.

Information Theoretic Methods for Variable Selection—A Review

“…Two interesting points arise from the study: strong performance of , which was the second best and co-winner in those cases, with respect to balanced accuracy (BA) and the number of feature chosen, with a strong performance of (winner in the case of Gisette and together with co-winner in the case of Madelon). The overall strong performance of was also confirmed in [ 68 , 69 ]. The second important observation is the failure of performance of , which due to scarcity of data, failed to detect conditionally dependent variables very early.…”

“…They are used to construct tests of conditional independence with approximate control of probability of false signals. The problem of existence of interactions can be similarly approached using results in [ 69 ].…”

“…This, however, may lead to a large number of false signals when the pertaining test is employed as the overall independence is only a very special situation when vanishes. It is shown in [ 69 ] that when are independent of Y , similarly to other measures of conditional dependence discussed in Section 4.1 , the approximate distribution is weighted chi-square distribution. Its complete description has been given for and having three values and Y being binary, which includes typical GWAS situation of two loci with two co-dominant alleles.…”

Information Theory in Computational Biology: Where We Stand Today

“…Although many definitions of statistical interactions and associated test statistics are proposed using information theory, the field would benefit from a unified definition and interpretation of statistical interactions. Additionally, often the underlying distributions of the test statistics under the null and the alternative hypothesis are unknown and more studies are needed as in [ 137 ], focusing on investigating the asymptotic behaviors of the estimators involved. Computing higher order test statistics like is computationally expensive as it necessitates entropy computations of all possible subsets of SNP combinations—for example, computing KWII for a set of two SNPs and a disease phenotype entails entropy computations with three random variables.…”