On the inconsistency of ℓ 1-penalised sparse precision matrix estimation

Abstract

BackgroundVarious ℓ 1-penalised estimation methods such as graphical lasso and CLIME are widely used for sparse precision matrix estimation and learning of undirected network structure from data. Many of these methods have been shown to be consistent under various quantitative assumptions about the underlying true covariance matrix. Intuitively, these conditions are related to situations where the penalty term will dominate the optimisation.ResultsWe explore the consistency of ℓ 1-based methods for a class of bipartite graphs motivated by the structure of models commonly used for gene regulatory networks. We show that all ℓ 1-based methods fail dramatically for models with nearly linear dependencies between the variables. We also study the consistency on models derived from real gene expression data and note that the assumptions needed for consistency never hold even for modest sized gene networks and ℓ 1-based methods also become unreliable in practice for larger networks.ConclusionsOur results demonstrate that ℓ 1-penalised undirected network structure learning methods are unable to reliably learn many sparse bipartite graph structures, which arise often in gene expression data. Users of such methods should be aware of the consistency criteria of the methods and check if they are likely to be met in their application of interest.

Back to the Basics: Rethinking Partial Correlation Network Methodology

“…In these situations, the covariance matrix cannot be inverted due to singularity (Hartlap, Simon, & Schneider, 2007), which is overcome by the glasso method. Accordingly, most of the simulation work has focused on high-dimensional settings (n < p), where model selection consistency is not typically evaluated in more common asymptotic settings (n → ∞) (Ha & Sun, 2014;Heinävaara, Leppä-aho, Corander, & Honkela, 2016;Peng, Wang, Zhou, & Zhu, 2009). Further, in behavioral science applications, the majority of network models are t in low-dimensional settings (p n) (Costantini et al, 2015;Rhemtulla et al, 2016).…”

Selected proceedings of Machine Learning in Systems Biology: MLSB 2016

“…Two additional papers describe aspects of (unsupervised) network reconstruction [3,4]. Affeldt et al…”

“…The basic idea here is to first identify related variables, and then in a second step perform multiple parallel local network reconstructions from which a global network is inferred. The second contribution related to network construction, Heinävaara et al [4], describes aspects of L1-penalised sparse precision matrix estimation. L1-regularisation is often applied in network reconstruction, and this manuscript demonstrates that it is important to check whether the conditions of consistency are likely to be met by the dataset and the problem at hand.…”

“…Two additional papers describe aspects of (unsupervised) network reconstruction [3,4]. Affeldt et al [3] present a consensus method based on spectral decomposition.…”

“…Their approach adaptively rescales genomic distances in order to enable clustering regions of interest with similar shapes. Park et al[2] apply association rule mining in order to find differential combinatorial chromatin modification patterns.Two additional papers describe aspects of (unsupervised) network reconstruction [3,4]. Affeldt et al…”