Optimal rates of convergence for covariance matrix estimation

Abstract

Covariance matrix plays a central role in multivariate statistical analysis. Significant advances have been made recently on developing both theory and methodology for estimating large covariance matrices. However, a minimax theory has yet been developed. In this paper we establish the optimal rates of convergence for estimating the covariance matrix under both the operator norm and Frobenius norm. It is shown that optimal procedures under the two norms are different and consequently matrix estimation under the operator norm is fundamentally different from vector estimation. The minimax upper bound is obtained by constructing a special class of tapering estimators and by studying their risk properties. A key step in obtaining the optimal rate of convergence is the derivation of the minimax lower bound. The technical analysis requires new ideas that are quite different from those used in the more conventional function/sequence estimation problems.Comment: Published in at http://dx.doi.org/10.1214/09-AOS752 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org

Optimal rates of convergence for estimating Toeplitz covariance matrices

“…Through a circulant matrix, a new estimatorΣ N ew can be constructed such that it is positive semidefinite, Toeplitz and attains the upper bound in (9). The construction is as follows.…”

“…The tapering estimatorΣ k * in (9) is not guaranteed to be positive semidefinite for a given sample. By using results on circulant matrices, one can construct a new estimatorΣ N ew based onΣ k * such thatΣ N ew is positive semidefinite, Toeplitz and attains the upper bound in Equation (9).…”

“…By using results on circulant matrices, one can construct a new estimatorΣ N ew based onΣ k * such thatΣ N ew is positive semidefinite, Toeplitz and attains the upper bound in Equation (9). See Section 5 for details.…”

Random matrix theory in statistics: A review

“…Related regularization methods have been proposed by Pourahmadi (2003, 2009). Cai et al (2010) proved the minimax rates for tapering estimators of covariance and precision matrices in terms of the spectral norm loss. Bhattacharjee and Bose (2013) established convergence rates for banded and tapered estimates of large dimensional covariance matrices under weak dependence among the entries.…”

Positive-Definite ℓ1-Penalized Estimation of Large Covariance Matrices

“…The usual sample covariance matrix is optimal in the classical setting with large samples and fixed low dimensions (Anderson 1984), but it performs very poorly in the highdimensional setting (Marčenko and Pastur 1967;Johnstone 2001). In the recent literature, regularization techniques have been used to improve the sample covariance matrix estimator, including banding (Wu and Pourahmadi 2003;Bickel and Levina 2008a), tapering (Furrer and Bengtsson 2007;Cai, Zhang, and Zhou 2010), and thresholding (Bickel and Levina 2008b;El Karoui 2008;Rothman, Levina, and Zhu 2009). Banding or tapering is very useful when the variables have a natural ordering and off-diagonal entries of the target covariance matrix decay to zero as they move away from the diagonal.…”