Optimal rates of convergence for sparse covariance matrix estimation

Abstract

This paper considers estimation of sparse covariance matrices and establishes the optimal rate of convergence under a range of matrix operator norm and Bregman divergence losses. A major focus is on the derivation of a rate sharp minimax lower bound. The problem exhibits new features that are significantly different from those that occur in the conventional nonparametric function estimation problems. Standard techniques fail to yield good results, and new tools are thus needed. We first develop a lower bound technique that is particularly well suited for treating "two-directional" problems such as estimating sparse covariance matrices. The result can be viewed as a generalization of Le Cam's method in one direction and Assouad's Lemma in another. This lower bound technique is of independent interest and can be used for other matrix estimation problems. We then establish a rate sharp minimax lower bound for estimating sparse covariance matrices under the spectral norm by applying the general lower bound technique. A thresholding estimator is shown to attain the optimal rate of convergence under the spectral norm. The results are then extended to the general matrix $\ell_w$ operator norms for $1\le w\le \infty$. In addition, we give a unified result on the minimax rate of convergence for sparse covariance matrix estimation under a class of Bregman divergence losses.Comment: Published in at http://dx.doi.org/10.1214/12-AOS998 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Positive-Definite ℓ1-Penalized Estimation of Large Covariance Matrices

“…Define d = max j k I {σ j k =0} and assume that σ max is bounded by a fixed constant, then we can pick λ = O((log p/n) 1/2 ) to achieve the minimax optimal rate of convergence under the Frobenius norm as in Theorem 4 of Cai and Zhou (2012b) …”

“…On the other hand, thresholding is proposed for estimating permutationinvariant covariance matrices. Thresholding can be used to produce consistent covariance matrix estimators when the true covariance matrix is bandable (Bickel and Levina 2008b;Cai and Zhou 2012a). In this sense, thresholding is more robust than banding/tapering for real applications.…”

Testing additive assumptions on means of regular monitoring data: a multivariate nonstationary time series approach

“…The dependence measure (5.1) provides a useful counterpart of (2.2) for studying high-dimensional time series, and has been used by Chen, Xu, and Wu (2013) for estimating large covariance and precision matrices. In order to make statistical inference for high-dimensional objects, additional regularity conditions are usually needed among which sparsity is one of the most popular; see for example Bickel and Levina (2008b), Cai and Zhou (2012) and references therein. Corollary 1 states that the maximum absolute deviation and the integrated squared deviation considered in Sections 3.2 and 3.3 respectively will continue to have the desired asymptotic distributions provided that the matrix A is sparse in the sense of the following condition.…”

Robust Inference of Risks of Large Portfolios

“…Under the assumption that data points are mutually independent, many sample covariance based regularization methods, including banding (Bickel and Levina, 2008b), tapering (Cai et al, 2010), thresholding (Bickel and Levina, 2008a;Cai and Zhou, 2012), and factor structures (Fan et al, 2008;Agarwal et al, 2012;Hsu et al, 2011), have been proposed. They are further applied to study stationary time series data under vector autoregressive dependence (Loh and Wainwright, 2012;, mixing conditions (Pan and Yao, 2008;Fan et al, 2011aFan et al, , 2013Han and Liu, 2013), and physical dependence (Xiao and Wu, 2012;Chen et al, 2013).…”