Preventing over‐fitting in PLS calibration models of near‐infrared (NIR) spectroscopy data using regression coefficients

Abstract

a Selection of the number of latent variables (LVs) to include in a partial least squares (PLS) model is an important step in the data analysis. Inclusion of too few or too many LVs may lead to, respectively, under or over-fitting of the data and subsequently result in poor future model performance. One well-known sign of over-fitting is the appearance of noise in regression coefficients; this often takes the form of a reduction in apparent structure and the presence of sharp peaks with a high degree of directional oscillation, features which are usually estimated subjectively. In this work, a simple method for quantifying the shape and size of a regression coefficient is presented. This measure can be combined with an indicator of model bias (e.g. root mean square error) to aid in estimation of the appropriate number of LVs to include in a PLS model. The performance of the proposed method is evaluated on simulated and and real NIR spectroscopy datasets sets and compared with several existing methods.

Sum of ranking differences (SRD) to ensemble multivariate calibration model merits for tuning parameter selection and comparing calibration methods

“…Another measure recently studied to characterize model complexity is the jaggedness of the model vector [28] defined by…”

“…As noted in section 1, approaches have been developed to remove the potential ambiguity in determining the corner region of an L-curve by forming U-curves with the best tuning parameter value at the minimum allowing automatic tuning parameter selection [20,23,28]. Two specific merits to be evaluated with SRD in this study are generalized CV (GCV) [46], AIC [47], BIC [48], trace (X T X) + [21], and others [12,18,19].…”

“…Sensitivity refers to the degree of change in signal relative to a change in the quantity of analyte, e.g., in analytical chemistry, a system is sensitive if a small change in analyte concentration generates a large change in signal [13,17]. It follows then that at least two model merits, each trending in opposite directions, should be simultaneously evaluated in order to characterize the balance between under-and over-fitting [20,21,[24][25][26][27][28].…”

“…One is a graphical approach forming L-curves by plotting RMSEC (or RMSECV) against a model complexity or variance measure with the better models residing in the corner region of the resultant L shaped curve [3,20,21.24-26,29]. The RMSEC (or RMSECV) values have been scaled and combined with scaled model complexity values or variance measures to convert L-curves to U-curves allowing automatic model selection [20,28]. Different combinations of RMSEC with RMSECV values have been plotted against model complexity or variance measures to form other U-curves [23].…”

Atmospheric particulate matter characterization by Fourier Transform Infrared spectroscopy: a review of statistical calibration strategies for carbonaceous aerosol quantification in US measurement networks

“…Evaluation metrics are calculated on samples which have not been involved in the model-building process (Esbensen and Geladi, 2010). Examples of metrics include the minimum root-mean-square error of cross validation (RMSECV) (one of the most widely used metrics; Gowen et al, 2011), one standard deviation above RMSECV (Hastie et al, 2009), Wold's R criterion (Wold, 1978), coefficient of determination -value (van der Voet, 1994;Wiklund et al, 2007), among others. A suite of these metrics can also be considered simultaneously (Zhao et al, 2015).…”