Estimation of high-dimensional graphical models using regularized score matching

Abstract

Graphical models are widely used to model stochastic dependences among large collections of variables. We introduce a new method of estimating undirected conditional independence graphs based on the score matching loss, introduced by Hyvärinen (2005), and subsequently extended in Hyvärinen (2007). The regularized score matching method we propose applies to settings with continuous observations and allows for computationally efficient treatment of possibly non-Gaussian exponential family models. In the well-explored Gaussian setting, regularized score matching avoids issues of asymmetry that arise when applying the technique of neighborhood selection, and compared to existing methods that directly yield symmetric estimates, the score matching approach has the advantage that the considered loss is quadratic and gives piecewise linear solution paths under ℓ1 regularization. Under suitable irrepresentability conditions, we show that ℓ1-regularized score matching is consistent for graph estimation in sparse high-dimensional settings. Through numerical experiments and an application to RNAseq data, we confirm that regularized score matching achieves state-of-the-art performance in the Gaussian case and provides a valuable tool for computationally efficient estimation in non-Gaussian graphical models.

Differential network analysis: A statistical perspective

“…To circumvent the latter shortcoming, few authors have proposed estimation strategies similar to conditionally specified models that result in symmetric network estimates (see, e.g., Drton & Maathuis, 2017). An alternative strategy for bypassing the computation of the normalizing constant, which can be used to directly obtain symmetric network estimates, is the score matching approach of Lin, Drton, & Shojaie (2016). In this approach, the loss function is defined as the Fisher information distance between the gradients, with respect to observations x , of true and candidate log densities.…”

Structure Learning in Graphical Modeling

“…Closed form score matching estimators are available for any pairwise interaction model vw ) can be obtained by adding an 1 or group lasso penalty to the lossĴ. The resulting estimators of conditional independence graphs are studied by Lin et al (2016) who also treat nonnegative observations, by Janofsky (2015) who proposes a nonparametric exponential series approach, and by Sun et al (2015) who consider infinitedimensional exponential families. For Gaussian models, 1 -regularized score matching is a simple but state-of-the-art method.…”

Fast Bayesian Inference in Large Gaussian Graphical Models

“…We explore three scenarios where the data are (a) multivariate‐ t distributed, (b) Gaussian contaminated, and (c) log‐Gaussian distributed. Scenarios 1 and 2 are as in Lin et al (), whereas scenario 3 introduces more skewness. For each scenario, we fix and generate 50 datasets of size using the same four graphical structures (and inverse‐covariance matrices) considered in Section .…”