A modification of the bootstrapping soft shrinkage approach for spectral variable selection in the issue of over-fitting, model accuracy and variable selection credibility

Yan, Hong; Song, Xianfeng; Tian, Kuangda; Gao, Jingxian; Li, Qianqian; Xiong, Yan; Min, Shungeng

doi:10.1016/j.saa.2018.10.034

Cited by 20 publications

(21 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The MPA has been classified into single variable model population analysis and interval model population analysis. The former includes random frog (RF) [14], iteratively retains informative variables (IRIV) [15], variable iterative space shrinkage approach (VISSA) [16], iteratively variable subset optimization (IVSO) [17], CARS [18], stability competitive adaptive reweighted sampling (SCARS) [19], sampling error profile analysis LASSO (SEPA-LASSO) [20], BOSS [4] and SBOSS [5]; while the latter includes interval random frog (iRF) [21], interval variable iterative space shrinkage approach (iVISSA) [22], interval combination optimization (ICO) [23] and fisher optimal subspace shrinkage (FOSS) [6]. Moreover, selecting the variables on near-infrared spectroscopy by utilizing models that hybridize two or more different techniques was recommended in [12].…”

Section: Related Workmentioning

confidence: 99%

“…The BOSS has the lowest variables selected then both VCPA-IRIV and BOSS-IRVS models have the same number of the variable selected. For the protein dataset, From Figure 8, we could observe that VCPA-IRIV, BOSS, and the BOSS-IRVS methods select the combination of several groups that are chemical meaningful for data analysis of spectrum [5]. All the methods selected the intervals around 1680, 1800 and 2180 nm.…”

Section: A Corn Datasetmentioning

confidence: 99%

“…Second and due to the use of bootstrap sampling that is inappropriate for the dependent data, the BOSS selects fewer variables, which causes missing some informative wavelengths. Third, it cannot avoid over-fitting problem BOSS uses RC, which is susceptible to noises [5], [6]. Most recently, three different methods have been developed and out-performed BOSS method.…”

Section: Introductionmentioning

confidence: 99%

“…Most recently, three different methods have been developed and out-performed BOSS method. The first method is a modification of the bootstrapping soft shrinkage approach named new computational method stabilized bootstrapping soft shrinkage approach (SBOSS) [5], in which variables are selected by the index of stability of regression coefficients instead of regression coefficients absolute value. Second, fisher optimal subspace shrinkage (FOSS) [6] that splits variables into some intervals by the information from regression coefficients PLS model, then the weighted block bootstrap sampling (WBBS) is used to select intervals, and the mean of the absolute values of regression coefficients of the corresponding interval determines the weights of sub-intervals.…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

A Bootstrapping Soft Shrinkage Approach and Interval Random Variables Selection Hybrid Model for Variable Selection in Near-Infrared Spectroscopy

et al. 2020

View full text Add to dashboard Cite

show abstract

Section: Related Workmentioning

confidence: 99%

Section: A Corn Datasetmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

A Bootstrapping Soft Shrinkage Approach and Interval Random Variables Selection Hybrid Model for Variable Selection in Near-Infrared Spectroscopy

et al. 2020

View full text Add to dashboard Cite

show abstract

“…The thousands of spectral data are too complicated to establish a calibration and prediction model, and the calibration process is quite time-consuming, which is even worse than the prediction performance of the model. Therefore, variable selection is an essential step before establishing the calibration and prediction models [29][30][31][32].…”

Section: Introductionmentioning

confidence: 99%

Nondestructive Analysis of Internal Quality in Pears with a Self-Made Near-Infrared Spectrum Detector Combined with Multivariate Data Processing

2021

Foods

View full text Add to dashboard Cite

The consumption of pears has increased, thanks not only to their delicious and juicy flavor, but also their rich nutritional value. Traditional methods of detecting internal qualities (e.g., soluble solid content (SSC), titratable acidity (TA), and taste index (TI)) of pears are reliable, but they are destructive, time-consuming, and polluting. It is necessary to detect internal qualities of pears rapidly and nondestructively by using near-infrared (NIR) spectroscopy. In this study, we used a self-made NIR spectrum detector with an improved variable selection algorithm, named the variable stability and cluster analysis algorithm (VSCAA), to establish a partial least squares regression (PLSR) model to detect SSC content in snow pears. VSCAA is a variable selection method based on the combination of variable stability and cluster analysis to select the infrared spectrum variables. To reflect the advantages of VSCAA, we compared the classical variable selection methods (synergy interval partial least squares (SiPLS), genetic algorithm (GA), successive projections algorithm (SPA), and bootstrapping soft shrinkage (BOSS)) to extract useful wavelengths. The PLSR model, based on the useful variables selected by SiPLS-VSCAA, was optimal for measuring SSC in pears, and the correlation coefficient of calibration (Rc), root mean square error of cross validation (RMSECV), correlation coefficient of prediction (Rp), root mean square error of prediction (RMSEP), and residual predictive deviation (RPD) were 0.942, 0.198%, 0.936, 0.222%, and 2.857, respectively. Then, we applied these variable selection methods to select the characteristic wavelengths for measuring the TA content and TI value in snow pears. The prediction PLSR models, based on the variables selected by GA-BOSS to measure TA and that by GA-VSCAA to detect TI, were the best models, and the Rc, RMSECV, Rp and RPD were 0.931, 0.124%, 0.912, 0.151%, and 2.434 and 0.968, 0.080%, 0.968, 0.089%, and 3.775, respectively. The results showed that the self-made NIR-spectrum detector based on a portable NIR spectrometer with multivariate data processing was a good tool for rapid and nondestructive analysis of internal quality in pears.

show abstract

Rapid non‐destructive analysis of lignin using NIR spectroscopy and chemo‐metrics

Liu

et al. 2021

Food and Energy Security

View full text Add to dashboard Cite

Lignin plays an important role in the formation of stone cells in pears. However, the accumulation of lignin had adverse impact on the flavor and quality of the fruit. A rapid and accurate method for measuring the lignin content of pears is therefore required. An improved variables selection method called ‘the bootstrapping soft shrinkage combined with frequency and regression coefficient of variables (FRCBOSS)’ was therefore developed based on ‘the bootstrapping soft shrinkage (BOSS)’technique, to identify the characteristic wavelengths of near‐infrared (NIR) spectra for non‐destructive and rapid analysis of lignin. Sub‐models were generated by weighted bootstrap sampling (WBS) in the FRCBOSS method. For the BOSS method, the new weights of variables were determined only as the absolute values of regression coefficients of variables in each iteration. In contrast, the FRCBOSS method also considers the frequency of variables in variable space. Moreover, the FRCBOSS algorithm overcomes the disadvantage of BOSS in selecting variables, which could incorporate useful wavelengths that would otherwise be removed by the BOSS method. In addition, a range of different pre‐treatment methods were used for comparison in the detection of lignin in the Snow pears. These include Savitzky–Golay Smoothing (SG), Normalization (NORM), Standard Normal Variate (SNV), and 1st Derivative (D1), as well as a combination of these methods and the different variables selection method (SiPLS, SiPLS‐SPA, SiPLS‐CARS, SiPLS‐BOSS, and SiPLS‐FRCBOSS). The number of variables selected by FRCBOSS was a little larger than that selected by BOSS. The partial least square regression (PLSR) model based on the 19 variables selected by SiPLS‐FRCBOSS method had the best prediction ability, with a Rp value of prediction of 0.880 and a RMSEP value of 1.004%. We conclude that NIR diffuse reflectance spectroscopy technology combined with FRC‐BOSS is an accurate and useful tool for the non‐destructive and rapid determination of pear lignin contents.

show abstract

A modification of the bootstrapping soft shrinkage approach for spectral variable selection in the issue of over-fitting, model accuracy and variable selection credibility

Cited by 20 publications

References 37 publications

A Bootstrapping Soft Shrinkage Approach and Interval Random Variables Selection Hybrid Model for Variable Selection in Near-Infrared Spectroscopy

A Bootstrapping Soft Shrinkage Approach and Interval Random Variables Selection Hybrid Model for Variable Selection in Near-Infrared Spectroscopy

Nondestructive Analysis of Internal Quality in Pears with a Self-Made Near-Infrared Spectrum Detector Combined with Multivariate Data Processing

Rapid non‐destructive analysis of lignin using NIR spectroscopy and chemo‐metrics

Contact Info

Product

Resources

About