Covariance Estimation in Decomposable Gaussian Graphical Models

Abstract

Abstract-Graphical models are a framework for representing and exploiting prior conditional independence structures within distributions using graphs. In the Gaussian case, these models are directly related to the sparsity of the inverse covariance (concentration) matrix and allow for improved covariance estimation with lower computational complexity. We consider concentration estimation with the mean-squared error (MSE) as the objective, in a special type of model known as decomposable. This model includes, for example, the well known banded structure and other cases encountered in practice. Our first contribution is the derivation and analysis of the minimum variance unbiased estimator (MVUE) in decomposable graphical models. We provide a simple closed form solution to the MVUE and compare it with the classical maximum likelihood estimator (MLE) in terms of performance and complexity. Next, we extend the celebrated Stein's unbiased risk estimate (SURE) to graphical models. Using SURE, we prove that the MSE of the MVUE is always smaller or equal to that of the biased MLE, and that the MVUE itself is dominated by other approaches. In addition, we propose the use of SURE as a constructive mechanism for deriving new covariance estimators. Similarly to the classical MLE, all of our proposed estimators have simple closed form solutions but result in a significant reduction in MSE.

Graphical model driven methods in adaptive system identification

“…For N max c∈C |c|, the convergence ofĴ x G (N ) to J x is quite rapid assuming the algorithm has access to the correct graphical model G [181] (no attempt is made in this case to analyze the effect of an incorrect model). Thus, usingĴ x G (N ) ≈ J x in (5.3a),…”

“…If x is characterized by a graph G, then J x has a number of zeros, as shown by (2.28), which can be exploited to simplify its estimation. Indeed, an estimator that exploits this property to improve the inverse covariance matrix estimate has been developed [181], [182]. The problem under consideration is not inverse covariance estimation but linear system identification, which requires a more indirect approach, but this provides some initial insight.…”

“…This is operationally identical (although the notation is slightly different) to the zero fill-in operator defined in [181]. The operation of (4.3) for the cliques and separators of the graph of Figure 4-1a is shown in As a final comment, the requirement of decomposability is implicitly used in the augmented graph.…”

“…[181], provided the graphical model used to compute the inverse covariance matrix truly does characterize the data.…”

“…, x(N ) are realizations of x, a random variable that is characterized by a decomposable graphical model. This problem was extensively investigated by Wiesel et al [181], in which it was shown that, when x is characterized by a decomposable graph G, the ML estimate of the inverse covariance matrix is given bŷ…”