The Peculiar Shrinkage Properties of Partial Least Squares Regression

Butler, Neil A.; Denham, Michael C.

doi:10.1111/1467-9868.00252

Cited by 60 publications

(37 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Stone and Brooks (1990) suggests that the methodology, in particular the choice of optimisation criterion, is arbitrary. Later Butler and Denham (2000) point out some rather non-intuitive shrinkage properties of the PLS procedure. However in applications, it is empirically competitive with other similar statistical procedures such as ridge regression and principal components regression, for instance see Frank and Friedman (1993).…”

Section: Introductionmentioning

confidence: 99%

The Use of Score Tests for Inference on Variance Components

Verbeke¹,

Molenberghs²

2003

Biometrics

203

144

View full text Add to dashboard Cite

Summary Whenever inference for variance components is required, the choice between one‐sided and two‐sided tests is crucial. This choice is usually driven by whether or not negative variance components are permitted. For two‐sided tests, classical inferential procedures can be followed, based on likelihood ratios, score statistics, or Wald statistics. For one‐sided tests, however, one‐sided test statistics need to be developed, and their null distribution derived. While this has received considerable attention in the context of the likelihood ratio test, there appears to be much confusion about the related problem for the score test. The aim of this paper is to illustrate that classical (two‐sided) score test statistics, frequently advocated in practice, cannot be used in this context, but that well‐chosen one‐sided counterparts could be used instead. The relation with likelihood ratio tests will be established, and all results are illustrated in an analysis of continuous longitudinal data using linear mixed models.

show abstract

Section: Introductionmentioning

confidence: 99%

The Use of Score Tests for Inference on Variance Components

Verbeke¹,

Molenberghs²

2003

Biometrics

203

144

View full text Add to dashboard Cite

show abstract

“…Note that all shrinkage factors in OLS, PCR, and ridge regression are not greater than 1. However, this is not true for PLS regression, and some counterexamples are given in [146,176]. Furthermore, unlike OLS, PCR, and ridge regression, the shrinkage factors in PLS regression also depends on the response Y nonlinearly.…”

Section: ) 55mentioning

confidence: 99%

Multi-Label Dimensionality Reduction

Sun

2016

View full text Add to dashboard Cite

“…This shrinkage, due to the fact that PLS is an inverse calibration procedure, has been studied extensively by various authors [1,7,29,30]. An interesting property in the context of the assumed model (12) for the spectra is that with an infinite training set the regression vector (13) is optimal under a mean squared error of prediction criterion.…”

Section: Pls In the Presence Of Noisementioning

confidence: 99%

Partial least squares, Beer's law and the net analyte signal: statistical modeling and analysis

Nadler

Coifman

2005

Journal of Chemometrics

View full text Add to dashboard Cite

Partial least squares (PLS) is one of the most common regression algorithms in chemistry, relating input-output samples (x i , y i ) by a linear multivariate model. In this paper we analyze the PLS algorithm under a specific probabilistic model for the relation between x and y. Following Beer's law, we assume a linear mixture model in which each data sample (x, y) is a random realization from a joint probability distribution where x is the sum of k components multiplied by their respective characteristic responses, and each of these components is a random variable. We analyze PLS on this model under two idealized settings: one is the ideal case of noise-free samples and the other is the case of an infinite number of noisy training samples. In the noise-free case we prove that, as expected, the regression vector computed by PLS is, up to normalization, the net analyte signal. We prove that PLS computes this vector after at most k iterations, where k is the total number of components. In the case of an infinite training set corrupted by unstructured noise, we show that PLS computes a final regression vector which is not in general purely proportional to the net analyte signal vector, but has the important property of being optimal under a mean squared error of prediction criterion. This result can be viewed as an asymptotic optimality of PLS in the limit of a very large but finite training set.

show abstract

The Peculiar Shrinkage Properties of Partial Least Squares Regression

Cited by 60 publications

References 6 publications

The Use of Score Tests for Inference on Variance Components

The Use of Score Tests for Inference on Variance Components

Multi-Label Dimensionality Reduction

Partial least squares, Beer's law and the net analyte signal: statistical modeling and analysis

Contact Info

Product

Resources

About