Pretreating near infrared spectra with fractional order Savitzky–Golay differentiation (FOSGD)

Zheng, Kaiyi; Zhang, Xuan; Tong, Peijin; Yao, Yuan; Du, Yukou

doi:10.1016/j.cclet.2014.10.023

Cited by 35 publications

(18 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The selection of 5 obviously caused underfitting, and RMSEC and RMSEP were higher than those using 15 as the n LVs selected by SEPA. Meanwhile, compared with the results of Zheng et al (Table ), the model performance described in this paper had good performance. Therefore, it can be concluded that SEPA makes it easy to build a model with better and robust prediction ability.…”

Section: Resultssupporting

confidence: 52%

Sampling error profile analysis (SEPA) for model optimization and model evaluation in multivariate calibration

Chen

Zhang

et al. 2017

Journal of Chemometrics

View full text Add to dashboard Cite

A novel method called sampling error profile analysis (SEPA) based on Monte Carlo sampling and error profile analysis is proposed for outlier detection, cross validation, pretreatment method and wavelength selection, and model evaluation in multivariate calibration. With the Monte Carlo sampling in SEPA, a number of submodels are prepared and the subsequent error profile analysis yields a median and a standard deviation of the root-mean-square error (RMSE) for the submodels. The median coupled with the standard deviation is an estimation of the RMSE that is more predictive and robust because it uses representative submodels produced by Monte Carlo sampling, unlike the normal method, which uses only 1 model. The error profile analysis also calculates skewness and kurtosis for an auxiliary judgment of the estimated RMSE, which is useful for model optimization and model evaluation. The proposed method is evaluated with 3 near-infrared datasets for wheat, corn, and tobacco. The results show that SEPA can diagnose outliers with more parameters, select more reasonable pretreatment method and wavelength points, and evaluate the model more accurately and precisely. Compared with the results reported in published papers, a better model could be obtained with SEPA concerning RMSECV, RMSEC, and RMSEP estimated with an independent prediction set. KEYWORDS model evaluation, Monte Carlo sampling, multivariate calibration, near-infrared, sampling error profile analysis (SEPA) | INTRODUCTIONMultivariate calibration is an important chemometric technique and effective tool for mining the intrinsic quantitative relations between spectra and the properties of samples of interest. In common research and practical applications, where the aim of calibration is to construct a robust and precise model, near-infrared (NIR) spectroscopy has gained increasing interest 1-4 in quantitative and qualitative spectroscopic analyses. In NIR, spectroscopic analysis model optimization, such as selecting the number of latent variables (LVs), selecting the spectral pretreatment method and wavelength, and model evaluation, are significant concerns.Cross validation (CV) is a commonly used method for selecting the number of LVs (nLVs) and can be used for model optimization. In a typical CV, the calibration and CV sets must cross over in successive rounds such that each sample has a chance of being validated against. 5 Leave-one-out (LOO) CV is the simplest and most commonly used method, but it often causes overfitting and underestimations of the true predictive error. 6-8 Then, K-fold CV was proposed to resolve such problems 9,10 ; in this process, samples are stratified prior to being split into K-folds. Stratification is the process of rearranging the data as to ensure that each fold is a good representative of the whole. The CV contributes a PRESS versus nLVs curve, where PRESS is the predicted residual sum of squares. With the curve, not only can the nLVs be selected but the optimized pretreatment method and wavelength selection can also be carried o...

show abstract

Section: Resultssupporting

confidence: 52%

Sampling error profile analysis (SEPA) for model optimization and model evaluation in multivariate calibration

Chen

Zhang

et al. 2017

Journal of Chemometrics

View full text Add to dashboard Cite

show abstract

“…Pre-processing of the spectrum is often required to reduce the effect of noise and enhance the spectral signature. Meanwhile, Savitzky-Golay differentiation is a commonly used spectral pre-treatment method, and in practice the first and second derivatives eliminate the interference of the baseline or background, improve sensitivity and detect and enhance minor or subtle spectral features [18,19]. Obtained spectra were continuum removed and normalized to enhance the spectral absorption features.…”

Section: Spectral Pre-processingmentioning

confidence: 99%

Estimate of Heavy Metals in Soil Using Combined Geochemistry and Field Spectroscopy in Miyi Mining Area

Ji¹,

Yao²,

Chen³

et al. 2018

Heavy Metals

View full text Add to dashboard Cite

Heavy metal-contaminated soil and water is a major environmental issue in the mining areas. However, as the heavy metals migrate frequently, the traditional method of estimating the soil's heavy metal content by field sampling and laboratory chemical analysis followed by interpolation is time-consuming and expensive. This chapter intends to use field hyperspectra to estimate the heavy metals in the soil in Bai-ma, De-sheng and YuanBaoshan mining areas, Miyi County, Sichuan Province. By analyzing the spectra of soil, the spectral features derived from the spectra of the soils can be found to build the models between these features and the contents of Mn and Co in the soil by using the linear regression method. The spectral features of Mn are 2142 and 2296 nm. The spectral features of Co are 1918, 1922 and 2205 nm. With these feature spectra, the best models to estimate the heavy metals in the study area can be built according to the maximal determination coefficients (R 2 ). The determination coefficients (R 2 ) of the models of retrieving Mn and Co in the soil are 0.645 and 0.8, respectively. The model significant indexes of Mn and Co are 2.04507E-05 and 7.73E-06. These results show that it is feasible to predict contaminated heavy metals in the soils during mining activities for soil remediation and ecological restoration by using the rapid and cost-effective field spectroscopy.Keywords: contaminated heavy metals in the soils, spectral measured, spectral analysis Heavy Metals 136

show abstract

“…He pointed out that the accuracy of infrared spectral data after fractional differential processing was improved compared with the integer-order operation. (9) Zhang et al applied the fractional differential in the pretreatment of hyperspectral data of saline soils, and indicated that it was desirable to use fractional differentials to excavate the potential information of soil spectra data. (10) At present, there are few reports on the estimation of total potassium content in soils using the fractional differential algorithm.…”

Section: Introductionmentioning

confidence: 99%

Quantitative Inversion Model of Total Potassium in Desert Soils Based on Multiple Regression Combined with Fractional Differential

Tian¹,

Zhao²,

Xiong³

et al. 2018

Sensors and Materials

View full text Add to dashboard Cite

Pretreating near infrared spectra with fractional order Savitzky–Golay differentiation (FOSGD)

Cited by 35 publications

References 24 publications

Sampling error profile analysis (SEPA) for model optimization and model evaluation in multivariate calibration

Sampling error profile analysis (SEPA) for model optimization and model evaluation in multivariate calibration

Estimate of Heavy Metals in Soil Using Combined Geochemistry and Field Spectroscopy in Miyi Mining Area

Quantitative Inversion Model of Total Potassium in Desert Soils Based on Multiple Regression Combined with Fractional Differential

Contact Info

Product

Resources

About