Cross-validation of component models: A critical look at current methods

Abstract

In regression, cross-validation is an effective and popular approach that is used to decide, for example, the number of underlying features, and to estimate the average prediction error. The basic principle of cross-validation is to leave out part of the data, build a model, and then predict the left-out samples. While such an approach can also be envisioned for component models such as principal component analysis (PCA), most current implementations do not comply with the essential requirement that the predictions should be independent of the entity being predicted. Further, these methods have not been properly reviewed in the literature. In this paper, we review the most commonly used generic PCA cross-validation schemes and assess how well they work in various scenarios.

Cross-validation in PCA models with the element-wise k-fold (ekf) algorithm: Practical aspects

“…Thus, if the decrease of PRESS when adding the a-th PC is lower than the threshold, the PC is discarded and the model selected contains a − 1 PCs. Also, the curve can be corrected with the degrees of freedom consumed [3].…”

“…The rkf has been criticized because the PCA estimates are computed using the actual values as input [3]. Since there is not independence between actual values and estimates, the modelling error computed in rkf cannot be considered purely prediction error.…”

“…The corresponding PRESS is obtained from the error computed between estimates and actual values following the equivalent expression to (2). Among the several cross-validatory algorithms proposed [6,3], those by Wold [2] and Eastment and Krzanowski [7] are the most cited and influent ones.…”

“…Thus, using the same calibration data set, different number of PCs may be optimal for process monitoring and compression, for instance. Therefore, seeking a single approach to select the number of PCs in PCA for a general perspective [3] is an ill-defined goal.…”

Interpretation of partial least squares regression models by means of target projection and selectivity ratio plots

“…For this case, numerous PLS components are usually needed and the x-variables are usually not scaled to equal variance. Typically some kind of cross validation is used to optimize the predictive power of the regression model [17]. Discussion of which validation procedure to use is outside the scope of the present work.…”