Natural order recovery for banded covariance models

Rolfs, Benjamin T.; Rajaratnam, Bala

doi:10.1109/sam.2012.6250512

Cited by 5 publications

(8 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…where λ is a regularization parameter, and C 1 is a matrix version of the L1 norm, namely a sum over the absolute values of the elements in C. It is important to emphasize that this approach is only applicable to chordal graphical models and it assumes that the perfect order of the variables is known a priori. Recent developments in high dimensional covariance estimation provide data-driven methods for identifying this order and structure [30], [28]. We leave this topic as a possible direction for future research.…”

Section: Convexity In Chordal Mggdmentioning

confidence: 99%

See 1 more Smart Citation

Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models

Zhang

Wiesel

Greco

2013

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

We consider covariance estimation in the multivariate generalized Gaussian distribution (MGGD) and elliptically symmetric (ES) distribution. The maximum likelihood optimization associated with this problem is non-convex, yet it has been proved that its global solution can be often computed via simple fixed point iterations. Our first contribution is a new analysis of this likelihood based on geodesic convexity that requires weaker assumptions. Our second contribution is a generalized framework for structured covariance estimation under sparsity constraints. We show that the optimizations can be formulated as convex minimization as long the MGGD shape parameter is larger than half and the sparsity pattern is chordal. These include, for example, maximum likelihood estimation of banded inverse covariances in multivariate Laplace distributions, which are associated with time varying autoregressive processes.

show abstract

Section: Convexity In Chordal Mggdmentioning

confidence: 99%

“…Our results are also applicable to the case of unknown sparsity pattern, i.e., structure learning via sparsity inducing penalties, but require prior knowledge of the perfect order. Recent works on structure learning in directed acyclic graphs provide data driven techniques for learning this order [30], [28].…”

Section: Introductionmentioning

confidence: 99%

Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models

Zhang

Wiesel

Greco

2013

IEEE Trans. Signal Process.

View full text Add to dashboard Cite

show abstract

“…Using the definition of ϵ in (7) with Corollary 1, we obtain the result below on the convergence of estimates.…”

Section: A Robust Sample Covariancementioning

confidence: 92%

“…Assuming that the underlying relationships follow Gaussian distributions, techniques such as graphical lasso [4]- [6] have been developed to tackle the problem by incorporating sparsity-promoting penalties. Different optimization methods have been proposed to solve the graphical lasso problem, including coordinate descent [5], proximal methods [7], [8], alternating minimization methods [9], [10], and Newtonconjugate gradient methods [11].…”

Section: Introductionmentioning

confidence: 99%

Robust Online and Distributed Mean Estimation Under Adversarial Data Corruption

Tong¹,

Sundaram²

2022

2022 IEEE 61st Conference on Decision and Control (CDC)

View full text Add to dashboard Cite

Gaussian graphical models are widely used to represent correlations among entities but remain vulnerable to data corruption. In this work, we introduce a modified trimmedinner-product algorithm to robustly estimate the covariance in an online scenario even in the presence of arbitrary and adversarial data attacks. At each time step, data points, drawn nominally independently and identically from a multivariate Gaussian distribution, arrive. However, a certain fraction of these points may have been arbitrarily corrupted. We propose an online algorithm to estimate the sparse inverse covariance (i.e., precision) matrix despite this corruption. We provide the error-bound and convergence properties of the estimates to the true precision matrix under our algorithms.

show abstract

“…Doing sparse graph recovery based on the graphical lasso formulation and its variants have been extensively studied and a recent survey [31] provides a good primer into these approaches. The traditional optimization based algorithms like BCD [2], G-ISTA [28], BigQUIC [18], GGMncv [48], MissGlasso [43] and many others have been designed for a range of requirements like scaling to large number of features, handling missing values, including non-convex penalties etc. The TeraLasso [14] and the Sylvester Graphical Lasso or SyGlasso model [47] are tensor based approaches and have recently garnered more interest.…”

Section: Related Methodsmentioning

confidence: 99%

Neural Graph Revealers

Shrivastava¹,

Chajewska²

2023

Preprint

View full text Add to dashboard Cite

Sparse graph recovery methods work well where the data follows their assumptions but often they are not designed for doing downstream probabilistic queries. This limits their adoption to only identifying connections among the input variables. On the other hand, the Probabilistic Graphical Models (PGMs) assume an underlying base graph between variables and learns a distribution over them. PGM design choices are carefully made such that the inference & sampling algorithms are efficient. This brings in certain restrictions and often simplifying assumptions. In this work, we propose Neural Graph Revealers (NGRs), that are an attempt to efficiently merge the sparse graph recovery methods with PGMs into a single flow. The problem setting consists of an input data X with D features and M samples and the task is to recover a sparse graph showing connection between the features and learn a probability distribution over the D at the same time. NGRs view the neural networks as a 'glass box' or more specifically as a multitask learning framework. We introduce 'Graph-constrained path norm' that NGRs leverage to learn a graphical model that captures complex non-linear functional dependencies between the features in the form of an undirected sparse graph. Furthermore, NGRs can handle multimodal inputs like images, text, categorical data, embeddings etc. which is not straightforward to incorporate in the existing methods. We show experimental results of doing sparse graph recovery and probabilistic inference on data from Gaussian graphical models and a multimodal infant mortality dataset by Centers for Disease Control and Prevention.

show abstract

Natural order recovery for banded covariance models

Cited by 5 publications

References 9 publications

Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models

Multivariate Generalized Gaussian Distribution: Convexity and Graphical Models

Robust Online and Distributed Mean Estimation Under Adversarial Data Corruption

Neural Graph Revealers

Contact Info

Product

Resources

About