Bootstrap confidence sets for spectral projectors of sample covariance

Naumov, Alexey; Spokoiny, Vladimir; Ulyanov, V. V.

doi:10.1007/s00440-018-0877-2

Cited by 30 publications

(34 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Moreover, we theoretically justify this method and derive finite sample bounds on the corresponding coverage probability. Contrary to [24,27], the obtained results are valid for non-Gaussian data: the main assumption that we impose is the concentration of the sample covariance Σ in a vicinity of Σ * . Numerical simulations illustrate good performance of the proposed procedure even on non-Gaussian data in a rather challenging regime.MSC 2010 subject classifications: Primary 62F15, 62H25, 62G20; secondary 62F25.I.…”

mentioning

confidence: 87%

“…The use of the classical conjugated Wishart prior helps not only to build a numerically efficient procedure but also to establish precise finite sample results for the posterior credible sets under mild and general assumptions on the data distribution. The key observation here is that, similarly to the bootstrap approach of [27], the credible level sets for the posterior are nearly elliptic, and the corresponding posterior probability can be approximated by a generalized chi-squared-type distribution. This allows to apply the recent "large ball probability" bounds on Gaussian comparison and Gaussian anti-concentration from [16].…”

Section: Introductionmentioning

confidence: 95%

“…Under the condition that the spectral gap is sufficiently large, [27] approximated the distribution of n P r − P * r 2 2 by the distribution of a Gaussian quadratic form ξ 2 with ξ ∼ N (0, Γ * r ) and Γ * r is a block-matrix of the form…”

Section: Setup and Problemmentioning

confidence: 99%

“…This allows to apply the recent "large ball probability" bounds on Gaussian comparison and Gaussian anti-concentration from [16]. Moreover, in the contrary to the latter paper [27], we do not require a Gaussian distribution of the data. Our results claim that the posterior credible sets can be used as frequentist confidence regions even under a possible model misspecification when the true data generation measure is not Gaussian.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Bayesian inference for spectral projectors of the covariance matrix

Silin¹,

Spokoiny²

2018

Electron. J. Statist.

Self Cite

View full text Add to dashboard Cite

Let X 1 , . . . , Xn be an i.i.d. sample in R p with zero mean and the covariance matrix Σ * . The classical PCA approach recovers the projector P * J onto the principal eigenspace of Σ * by its empirical counterpart P J . Recent paper [24] investigated the asymptotic distribution of the Frobenius distance between the projectors P J − P * J 2 , while [27] offered a bootstrap procedure to measure uncertainty in recovering this subspace P * J even in a finite sample setup. The present paper considers this problem from a Bayesian perspective and suggests to use the credible sets of the pseudo-posterior distribution on the space of covariance matrices induced by the conjugated Inverse Wishart prior as sharp confidence sets. This yields a numerically efficient procedure. Moreover, we theoretically justify this method and derive finite sample bounds on the corresponding coverage probability. Contrary to [24,27], the obtained results are valid for non-Gaussian data: the main assumption that we impose is the concentration of the sample covariance Σ in a vicinity of Σ * . Numerical simulations illustrate good performance of the proposed procedure even on non-Gaussian data in a rather challenging regime.MSC 2010 subject classifications: Primary 62F15, 62H25, 62G20; secondary 62F25.I. Silin and V. Spokoiny /Bayesian inference for spectral projectors 2 trix:Usually one estimates the true unknown covariance by the sample covariance matrix, given byQuantifying the quality of approximation of Σ * by Σ is one of the most classical problems in statistics. Surprisingly, a number of deep and strong results in this area appeared quite recently. The progress is mainly due to Bernstein type results on the spectral norm Σ − Σ * ∞ in the random matrix theory, see, for instance, [22,29,31,33,1]. It appears that the quality of approximation is of order n −1/2 while the dimensionality p only enters logarithmically in the error bound. This allows to apply the results even in the cases of very high data dimension.Functionals of the covariance matrix also arise in applications frequently. For instance, eigenvalues are well-studied in different regimes, see [26,9,19,34] and many more references therein. The Frobenius norm and other l r -norms of covariance matrix are of great interest in financial applications; see, e.g. [10].Much less is known about the quality of estimation of a spectral projector which is a nonlinear functional of the covariance matrix. However, such objects arise in dimension reduction methods, manifold learning and spectral methods in community detection, see [11] and references therein for an overview of problems where spectral projectors play crucial role. Special attention should be focused on the Principal Component Analysis (PCA), probably the most famous dimension reduction method. Nowadays PCA-based methods are actively used in deep networking architecture [17] and finance [12], along with other applications. Over the past decade huge progress was achieved in theoretical guarantees for sparse PCA in high dimensions...

show abstract

mentioning

confidence: 87%

Section: Introductionmentioning

confidence: 95%

Section: Setup and Problemmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Bayesian inference for spectral projectors of the covariance matrix

Silin¹,

Spokoiny²

2018

Electron. J. Statist.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Bayesian Neural Networks produce a probabilistic relationship between the network input and output [16,17], but often suffer from tractability issues. Ensemble based approaches, bootstrapping, and Monte Carlo based approaches have also been proposed, for example [18,19,20,21]. While such approaches can produce calibrated prediction intervals, they often require training and testing a multitude of different individual networks which considerably increases the associated time and computational costs [22].…”

Section: Introductionmentioning

confidence: 99%

Calibrated Prediction Intervals for Neural Network Regressors

Keren¹,

Cummins²,

Schuller³

2018

IEEE Access

View full text Add to dashboard Cite

Ongoing developments in neural network models are continually advancing the state of the art in terms of system accuracy. However, the predicted labels should not be regarded as the only core output; also important is a well-calibrated estimate of the prediction uncertainty. Such estimates and their calibration are critical in many practical applications. Despite their obvious aforementioned advantage in relation to accuracy, contemporary neural networks can, generally, be regarded as poorly calibrated and as such do not produce reliable output probability estimates. Further, while post-processing calibration solutions can be found in the relevant literature, these tend to be for systems performing classification. In this regard, we herein present two novel methods for acquiring calibrated predictions intervals for neural network regressors: empirical calibration and temperature scaling. In experiments using different regression tasks from the audio and computer vision domains, we find that both our proposed methods are indeed capable of producing calibrated prediction intervals for neural network regressors with any desired confidence level, a finding that is consistent across all datasets and neural network architectures we experimented with. In addition, we derive an additional practical recommendation for producing more accurate calibrated prediction intervals. We release the source code implementing our proposed methods for computing calibrated predicted intervals. The code for computing calibrated predicted intervals is publicly available 1 .

show abstract

Nonasymptotic One- and Two-Sample Tests in High Dimension with Unknown Covariance Structure

Blanchard

Fermanian

2023

Springer Proceedings in Mathematics &Amp; Statistics

View full text Add to dashboard Cite

Bootstrap confidence sets for spectral projectors of sample covariance

Cited by 30 publications

References 23 publications

Bayesian inference for spectral projectors of the covariance matrix

Bayesian inference for spectral projectors of the covariance matrix

Calibrated Prediction Intervals for Neural Network Regressors

Nonasymptotic One- and Two-Sample Tests in High Dimension with Unknown Covariance Structure

Contact Info

Product

Resources

About