High-dimensional structure estimation in Ising models: Local separation criterion

Abstract

We consider the problem of high-dimensional Ising (graphical) model selection. We propose a simple algorithm for structure estimation based on the thresholding of the empirical conditional variation distances. We introduce a novel criterion for tractable graph families, where this method is efficient, based on the presence of sparse local separators between node pairs in the underlying graph. For such graphs, the proposed algorithm has a sample complexity of n = Ω(J −2 min log p), where p is the number of variables, and Jmin is the minimum (absolute) edge potential in the model. We also establish nonasymptotic necessary and sufficient conditions for structure estimation.

Greedy Learning of Graphical Models with Small Girth

“…These can be broadly divided into three classes -search based, convexoptimization based, and greedy methods. Search based algorithms like local independence test by Bresler et al in [10] and the conditional variation distance thresholding (CVDT) by Anandkumar et al in [13] try to find the smallest set of nodes through exhaustive search, conditioned on which either a given node is independent of rest of the nodes in the graph, or a pair of nodes are independent of each other. These algorithms although have a fairly good sample complexity, but due to exhaustive search they have a high computation complexity.…”

“…Many different forms of correlation decay have been assumed in MRF learning algorithms [10], [12], [13]. We assume a weak form of correlation decay similar to the weak spatial mixing assumption in [19].…”

“…Lemmas 4, 5, 6 also follow from Azuma's concentration inequality and the proofs are omitted due to space constraint. Although the sample complexities of RecGreedy( ) and F bGreedy( , α) algorithms are slightly more than other nongreedy algorithms [9], [10], [13], the main appeal of these greedy algorithms lie in their low computation complexity.…”

“…Search based graphical model learning algorithms like the Local Independence Test (LIT) by Bresler et al [10] and the Conditional Variation Distance Thresholding (CVDT) by Anandkumar et al [13] generally have good sample complexity but suffer due to their high computation complexity. As we will see the RecGreedy( ) and F bGreedy( , α) algorithms have slightly more sample complexity but significantly lower computational complexity than the search based algorithms.…”

New Theory and Algorithms for Scalable Data Fusion

“…Other related work on graphical model selection for discrete graphs includes the classic Chow-Liu algorithm for trees [12], nodewise logistic regression for discrete models with pairwise interactions [40,18], and techniques based on conditional entropy or mutual information [9,4]. Our main contribution is to present a clean and surprising result on a simple link between the inverse covariance matrix and edge structure of a discrete model, which may be used to derive inference algorithms applicable even to data with systematic corruptions.…”