Estimating Large Precision Matrices via Modified Cholesky Decomposition

Lee, Kyoungjae; Lee, Jae Yong

doi:10.48550/arxiv.1707.01143

Cited by 6 publications

(18 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lee and Lee [71] considered a class of bandable precision matrices in the highdimensional setup, using a modified Cholesky decomposition (MCD) approach Ω = (I p − A) T D −1 (I p − A), where D is a diagonal matrix, and A is lower-triangular with zero diagonal entries. This results in the k-banded Cholesky prior with nonzero entries of A getting the improper uniform prior and the diagonal entries a truncated polynomial prior.…”

Section: Banding and Other Special Sparsity Patternsmentioning

confidence: 99%

Bayesian inference in high-dimensional models

Banerjee,

Castillo,

Ghosal

2021

Preprint

View full text Add to dashboard Cite

Models with dimension more than the available sample size are now commonly used in various applications. A sensible inference is possible using a lowerdimensional structure. In regression problems with a large number of predictors, the model is often assumed to be sparse, with only a few predictors active. Interdependence between a large number of variables is succinctly described by a graphical model, where variables are represented by nodes on a graph and an edge between two nodes is used to indicate their conditional dependence given other variables. Many procedures for making inferences in the high-dimensional setting, typically using penalty functions to induce sparsity in the solution obtained by minimizing a loss function, were developed. Bayesian methods have been proposed for such problems more recently, where the prior takes care of the sparsity structure. These methods have the natural ability to also automatically quantify the uncertainty of the inference through the posterior distribution. Theoretical studies of Bayesian procedures in high-dimension have been carried out recently. Questions that arise are, whether the posterior distribution contracts near the true value of the parameter at the minimax optimal rate, whether the correct lower-dimensional structure is discovered with high posterior probability, and whether a credible region has adequate frequentist coverage. In this paper, we review these properties of Bayesian and related methods for several high-dimensional models such as many normal means problem, linear regression, generalized linear models, Gaussian and non-Gaussian graphical models. Effective computational approaches are also discussed.

show abstract

Section: Banding and Other Special Sparsity Patternsmentioning

confidence: 99%

Bayesian inference in high-dimensional models

Banerjee,

Castillo,

Ghosal

2021

Preprint

View full text Add to dashboard Cite

show abstract

“…Since we have bandwidth selection consistency (Theorem 3.1), we suggest using the posterior mode to estimate the true bandwidth k 0 . We chose the bandwidth test of An et al (2014) as a frequentist competitor and the bandwidth selection procedures of Banerjee and Ghosal (2014) and Lee and Lee (2017) as Bayesian competitors. Significance levels for bandwidth tests in An et al (2014) were varied α = 0.001, 0.005, 0.01, but only the result with α = 0.01 are reported since they gave similar results.…”

Section: Comparison With Other Bandwidth Testsmentioning

confidence: 99%

“…Significance levels for bandwidth tests in An et al (2014) were varied α = 0.001, 0.005, 0.01, but only the result with α = 0.01 are reported since they gave similar results. For Banerjee and Ghosal (2014) and Lee and Lee (2017), we used the prior π(k) ∝ exp(−k 4 ) as they suggested. Note that these Bayesian procedures do not guarantee the bandwidth selection consistency.…”

Section: Comparison With Other Bandwidth Testsmentioning

confidence: 99%

“…They derived the posterior convergence rate of the precision matrix under the G-Wishart prior (Roverato;2000). Lee and Lee (2017) considered a similar class to that of Banerjee and Ghosal (2014), but assumed bandable Cholesky factors instead of bandable precision matrices. They showed the posterior convergence rates of the precision matrix as well as the minimax lower bounds.…”

Section: Introductionmentioning

confidence: 99%

“…The key difference from Lee and Lee (2017) is on the choice of prior distributions which will be introduced in Section 2.3. In addition, we focus on bandwidth selection and tests, while Lee and Lee (2017) mainly studied the convergence rates of the precision matrix for a given or fixed bandwidth.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bayesian Bandwidth Test and Selection for High-dimensional Banded Precision Matrices

Lee¹,

Lin²

2020

Bayesian Anal.

Self Cite

View full text Add to dashboard Cite

Assuming a banded structure is one of the common practice in the estimation of highdimensional precision matrix. In this case, estimating the bandwidth of the precision matrix is a crucial initial step for subsequent analysis. Although there exist some consistent frequentist tests for the bandwidth parameter, bandwidth selection consistency for precision matrices has not been established in a Bayesian framework. In this paper, we propose a prior distribution tailored to the bandwidth estimation of high-dimensional precision matrices. The banded structure is imposed via the Cholesky factor from the modified Cholesky decomposition. We establish the strong model selection consistency for the bandwidth as well as the consistency of the Bayes factor. The convergence rates for Bayes factors under both the null and alternative hypotheses are derived which yield similar order of rates. As a by-product, we also proposed an estimation procedure for the Cholesky factors yielding an almost optimal order of convergence rates. Two-sample bandwidth test is also considered, and it turns out that our method is able to consistently detect the equality of bandwidths between two precision matrices. The simulation study confirms that our method in general outperforms the existing frequentist and Bayesian methods.

show abstract

Sparse inverse covariance matrix estimation via the $ \newcommand{\e}{{\rm e}} \ell_{0}$ -norm with Tikhonov regularization

Liu

Zhang

2019

Inverse Problems

View full text Add to dashboard Cite

Sparse inverse covariance matrix estimation is a fundamental problem in a Gaussian network model and has attracted much attention in the last decade. A widely used approach for estimating the sparse inverse covariance matrix is the regularized negative log-likelihood minimization, where the regularization term promotes the sparsity of the inverse covariance matrix. Although the -norm is the most popular sparsity promoting function due to its convexity, it may not result in sufficiently satisfied estimations. The -norm is the most natural function to measure sparsity and the -norm regularized negative log-likelihood minimization has been demonstrated to produce favorable results. However, when the sample covariance matrix is singular, the -norm problem has no optimal solution. In this paper, we propose to estimate the sparse inverse covariance matrix via simultaneously using the -norm and the Tikhonov regularization. The resulting minimization problem is guaranteed to have optimal solutions. To overcome the numerical difficulty caused by the -norm, we utilize the penalty decomposition approach. We prove the asymptotic approximation and exact approximation to local (resp., global) minimizers of the original problem by those of the penalty decomposition problem. Then, local minimizers of the penalty decomposition problem are characterized as fixed-points of a system of fixed-point equations with the help of proximity operators. Based on these theoretical results, we design for solving the proposed minimization problem a penalty decomposition fixed-point proximity algorithm and provide the convergence analysis of the algorithm. Numerical results indicate that the proposed algorithm outperforms the compared state-of-the-art algorithms in terms of accuracy.

show abstract

Estimating Large Precision Matrices via Modified Cholesky Decomposition

Cited by 6 publications

References 0 publications

Bayesian inference in high-dimensional models

Bayesian inference in high-dimensional models

Bayesian Bandwidth Test and Selection for High-dimensional Banded Precision Matrices

Sparse inverse covariance matrix estimation via the $ \newcommand{\e}{{\rm e}} \ell_{0}$ -norm with Tikhonov regularization

Contact Info

Product

Resources

About