Mining non-redundant high order correlations in binary data

Abstract

Many approaches have been proposed to find correlations in binary data. Usually, these methods focus on pair-wise correlations. In biology applications, it is important to find correlations that involve more than just two features. Moreover, a set of strongly correlated features should be non-redundant in the sense that the correlation is strong only when all the interacting features are considered together. Removing any feature will greatly reduce the correlation.In this paper, we explore the problem of finding non-redundant high order correlations in binary data. The high order correlations are formalized using multi-information, a generalization of pairwise mutual information. To reduce the redundancy, we require any subset of a strongly correlated feature subset to be weakly correlated. Such feature subsets are referred to as Non-redundant Interacting Feature Subsets (NIFS). Finding all NIFSs is computationally challenging, because in addition to enumerating feature combinations, we also need to check all their subsets for redundancy. We study several properties of NIFSs and show that these properties are useful in developing efficient algorithms. We further develop two sets of upper and lower bounds on the correlations, which can be incorporated in the algorithm to prune the search space. A simple and effective pruning strategy based on pair-wise mutual information is also developed to further prune the search space. The efficiency and effectiveness of our approach are demonstrated through extensive experiments on synthetic and real-life datasets.

Mining non-redundant diverse patterns: an information theoretic perspective

“…We will introduce some conclusions from Ref. [5] and show that our new bounds are tighter than those. Before stating those conclusions and their proofs, we introduce another formulations of the entropy of the discrete distributions.…”

“…[5], the authors explore the problem of finding non-redundant high order correlations in binary data. Both the algorithm in Ref.…”

“…Both the algorithm in Ref. [5] and ours adopt depth-first search and use serval strategies to prune the pattern search space, which are based on some bounds on entropy.…”

“…Those patterns have found broad applications in areas like association rule mining and classification [4,5].…”

CMI: An Information-Theoretic Contrast Measure for Enhancing Subspace Cluster and Outlier Detection

“…They avoid the limitations of the above paradigms, and focus on multiple projections with arbitrary dimensionality. Existing methods, however, rely on discretization of continuous dimensions [9,27] or only work with binary data [28] and/or discrete data [8].…”