Lower bounds for two-sample structural change detection in ising and Gaussian models

Abstract

The change detection problem is to determine if the Markov network structures of two Markov random fields differ from one another given two sets of samples drawn from the respective underlying distributions. We study the trade-off between the sample sizes and the reliability of change detection, measured as a minimax risk, for the important cases of the Ising models and the Gaussian Markov random fields restricted to the models which have network structures with p nodes and degree at most d, and obtain information-theoretic lower bounds for reliable change detection over these models. We show that for the Ising model, Ω d 2 (log d) 2 log p samples are required from each dataset to detect even the sparsest possible changes, and that for the Gaussian, Ω γ −2 log(p) samples are required from each dataset to detect change, where γ is the smallest ratio of off-diagonal to diagonal terms in the precision matrices of the distributions. These bounds are compared to the corresponding results in structure learning, and closely match them under mild conditions on the model parameters. Thus, our change detection bounds inherit partial tightness from the structure learning schemes in previous literature, demonstrating that in certain parameter regimes, the naive structure learning based approach to change detection is minimax optimal up to constant factors.capturing the dependencies, called the Markov network structure of the distribution, G(θ) = ([1 : p], E(θ)), where E = {(i, j) : θ ij = 0}. The Ising model and the Gaussian MRF are instances of MRFs for binary and real-valued random variables, respectively, and are employed in a variety of applications.An important problem in statistics is the inverse problem of determining the dependencies between a set of random variables given a set of samples drawn from their joint distributions. In the context of MRFs, let M be a class of MRFs as before, P ∈ M be parametrised by θ, and the random vector X = (X i ) i∈[1 : p] be distributed according to P . The inverse problem can be stated in two closely related ways -given n i.i.d. samples X n ∼ P ⊗n , one may wish to either learn θ, which is the model selection problem, or to learn G(θ), which is the structure learning problem. These problems are typically studied in the "high-dimensional setting," where n ≪ p, that is, the number of samples is much smaller than the number of variables involved. A large body of work has focussed on constructing structure learning schemes for the Gaussian and Ising models, including algorithms and regularised estimators, with a particular focus on the technical conditions and the number of samples required for consistent learning (see [Ana+12a; MB06; RWL10] and references therein for the Gaussian MRF, and [BM09; Bre15; RWL10; Ana+12b], and references therein for the Ising model). A relatively smaller set of papers has studied the hardness of structure learning in terms of simple graph properties, providing necessary and sufficient information-theoretic conditions on the number of samples required for r...

Testing Ising Models

“…One may also consider testing problems in settings involving Markov Chains, of which there has been interest in testing standard properties as well as domain specific ones (i.e., the mixing time) [BFF + 01, BV15, DDG18, HKS15, LP16, HKL + 17, BK18]. There have also been other recent works on learning and testing Ising models, in both the statistical and structural sense [GNS17,DMR18,BN18]. It remains to be seen which other multivariate distribution classes of interest allow us to bypass the curse of dimensionality.…”