A systematic study on the application of scatter-corrective and spectral-derivative preprocessing for multivariate prediction of soil organic carbon by Vis-NIR spectra

Dotto, André Carnieletto; Dalmolin, Ricardo Simão Diniz; Caten, Alexandre ten; Grunwald, Sabine

doi:10.1016/j.geoderma.2017.11.006

Cited by 203 publications

(124 citation statements)

References 69 publications

Supporting

Mentioning

113

Contrasting

Unclassified

Order By: Relevance

“…SVM algorithm has been widely applied to solve the complicated regression problems. Recently, Dotto et al [12] reviewed a series of the multivariate methods and investigated their influences on the SOC prediction. Their results indicated that SVM achieved the highest performance in the SOC estimation, among the nine commonly used multivariate methods.…”

Section: Discussionmentioning

confidence: 99%

“…Several preprocessing methods (i.e., data transformation, spectral smoothing, scatter corrections and spectral derivatives and so on), have been utilized to transform the reflectance spectra, reduce the instrumental noise, enhance spectral features and extract useful spectral information for subsequent modeling [10]. These spectral preprocessing techniques can be mainly divided into two categories: scatter-corrections and spectral-derivatives, according to Rinnan et al [11] and Dotto et al [12], but it should be noted that the accuracies of these preprocessing methods may vary from case to case. For spectral-derivative preprocessing, the first and second derivatives are commonly used to eliminate unwanted interference (e.g., removing baseline effects) exerting on the SOM estimation.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

Hong

Chen

et al. 2018

Remote Sensing

View full text Add to dashboard Cite

Visible and near-infrared (VIS-NIR) spectroscopy has been extensively applied to estimate soil organic matter (SOM) in the laboratory. However, if field/moist VIS-NIR spectra can be directly applied to estimate SOM, then much of the time and labor would be avoided. Spectral derivative plays an important role in eliminating unwanted interference and optimizing the estimation model. Nonetheless, the conventional integer order derivatives (i.e., the first and second derivatives) may neglect some detailed information related to SOM. Besides, the full-spectrum generally contains redundant spectral variables, which would affect the model accuracy. This study aimed to investigate different combinations of fractional order derivative (FOD) and spectral variable selection techniques (i.e., competitive adaptive reweighted sampling (CARS), elastic net (ENET) and genetic algorithm (GA)) to optimize the VIS-NIR spectral model of moist soil. Ninety-one soil samples were collected from Central China, with their SOM contents and reflectance spectra measured. Support vector machine (SVM) was applied to estimate SOM. Results indicated that moist spectra differed greatly from dried ground spectra. With increasing order of derivative, the spectral resolution improved gradually, but the spectral strength decreased simultaneously. FOD could provide a better tool to counterbalance the contradiction between spectral resolution and spectral strength. In full-spectrum SVM models, the most accurate estimation was achieved by SVM model based on 1.5-order derivative spectra, with validation R 2 = 0.79 and ratio of the performance to deviation (RPD) = 2.20. Of all models studied (different combinations of FOD and variable selection techniques), the highest validation model accuracy for SOM was achieved when applying 1.5 derivative spectra and GA method (validation R 2 = 0.88 and RPD = 2.89). Among the three variable selection techniques, overall, the GA method yielded the optimal predictability. However, due to its long computation time, one alternative was to use CARS method. The results of this study confirm that a suitable combination of FOD and variable selection can effectively improve the model performance of SOM in moist soil.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

Hong

Chen

et al. 2018

Remote Sensing

View full text Add to dashboard Cite

show abstract

“…In addition, the quantification of soil properties via spectroscopy is generated by building calibration models that correlate the spectra with the reference analytical values. The types of spectral preprocessing and multivariate methods influence the quantification of soil properties (Dotto et al, 2018). In this study, we intended to develop the AlradSpectra to facilitate and disseminate the use of soil spectroscopy technique.…”

Section: Discussionmentioning

confidence: 99%

“…In addition, spectroscopy in the visible (Vis: 400-700 nm), near infrared (NIR: 701-1100 nm), and short-wave infrared (SWIR: 1101-2500 nm) regions of the electromagnetic spectrum associated with chemometric methods has allowed the quantification of physical, chemical, and mineralogical soil properties (Viscarra Rossel and Behrens, 2010). This technique has become a well-established method to assess soil properties rapidly and accurately in the laboratory (Ben Dor et al, 2015), with the possibility of predicting several properties in just one spectral reading, facilitating data acquisition from large amounts of samples, and without the use of environmentally hazardous chemicals (Dotto et al, 2016(Dotto et al, , 2018Demattê et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

AlradSpectra: a Quantification Tool for Soil Properties Using Spectroscopic Data in R

Dotto

Dalmolin

Caten

et al. 2019

Rev. Bras. Ciênc. Solo

View full text Add to dashboard Cite

Soil reflectance spectroscopy has become an innovative method for soil property quantification supplying data for studies in soil fertility, soil classification, digital soil mapping, while reducing laboratory time and applying a clean technology. This paper describes the implementation of a Graphical User Interface (GUI) using R named AlradSpectra. It contains several tools to process spectroscopic data and generate models to predict soil properties. The GUI was developed to accomplish tasks such as perform a large range of spectral preprocessing techniques, implement several multivariate calibration methods, generate statistics assessment and graphical output, validate the models using independent dataset, and predict unknown variables using soil spectral data. AlradSpectra has four main modules: Import Data, Spectral Preprocessing, Modeling, and Prediction. The implementation of AlradSpectra is demonstrated by applying visible near-infrared reflectance spectroscopy for soil organic carbon (SOC) prediction. The data contains the value of SOC and Vis-NIR reflectance for 595 soil samples. The prediction statistic assessment of SOC was performed applying all spectral preprocessing and methods. The R 2 considering all models ranged from 0.54 to 0.80. In the partial least squares regression (PLSR) models, the performances were similar to multiple linear regression (MLR) and support vector machines (SVM). The lowest error in the SOC prediction was achieved by PLSR method with standard normal variate (SNV) preprocessing reaching an R 2 of 0.80, the smallest root mean square error (RMSE) of 0.47 %, and ratio of performance to inter-quartile distance (RPIQ) of 3.12. The capacity of performing multiple tasks, being free and open-source, easy to operate, and requiring no initial knowledge of R programming language are features that make AlradSpectra a useful tool to perform different modeling approaches and predict the desired soil variable.

show abstract

“…e next is spectra preprocessing where it has been used to reduce noise and extract useful information for developing a quantitative calibration model where it includes smoothing, normalization, scatter-correction, and Figure 1: Absorption band in near-infrared (NIR) [14]. International Journal of Analytical Chemistry derivatives [17]. is spectral pretreatment will help to lower the errors of estimation for the calibration model developed.…”

Section: Spectral Datamentioning

confidence: 99%

Monitoring of CO₂ Absorption Solvent in Natural Gas Process Using Fourier Transform Near-Infrared Spectrometry

Yusop

Norfadilah

Aishah

et al. 2020

International Journal of Analytical Chemistry

View full text Add to dashboard Cite

The analytical methods for the determination of the amine solvent properties do not provide input data for real-time process control and optimization and are labor-intensive, time-consuming, and impractical for studies of dynamic changes in a process. In this study, the potential of nondestructive determination of amine concentration, CO2 loading, and water content in CO2 absorption solvent in the gas processing unit was investigated through Fourier transform near-infrared (FT-NIR) spectroscopy that has the ability to readily carry out multicomponent analysis in association with multivariate analysis methods. The FT-NIR spectra for the solvent were captured and interpreted by using suitable spectra wavenumber regions through multivariate statistical techniques such as partial least square (PLS). The calibration model developed for amine determination had the highest coefficient of determination (R2) of 0.9955 and RMSECV of 0.75%. CO2 calibration model achieved R2 of 0.9902 with RMSECV of 0.25% whereas the water calibration model had R2 of 0.9915 with RMSECV of 1.02%. The statistical evaluation of the validation samples also confirmed that the difference between the actual value and the predicted value from the calibration model was not significantly different and acceptable. Therefore, the amine, CO2, and water models have given a satisfactory result for the concentration determination using the FT-NIR technique. The results of this study indicated that FT-NIR spectroscopy with chemometrics and multivariate technique can be used for the CO2 solvent monitoring to replace the time-consuming and labor-intensive conventional methods.

show abstract

A systematic study on the application of scatter-corrective and spectral-derivative preprocessing for multivariate prediction of soil organic carbon by Vis-NIR spectra

Cited by 203 publications

References 69 publications

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

AlradSpectra: a Quantification Tool for Soil Properties Using Spectroscopic Data in R

Monitoring of CO₂ Absorption Solvent in Natural Gas Process Using Fourier Transform Near-Infrared Spectrometry

Contact Info

Product

Resources

About

A systematic study on the application of scatter-corrective and spectral-derivative preprocessing for multivariate prediction of soil organic carbon by Vis-NIR spectra

Cited by 203 publications

References 69 publications

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

Combining Fractional Order Derivative and Spectral Variable Selection for Organic Matter Estimation of Homogeneous Soil Samples by VIS–NIR Spectroscopy

AlradSpectra: a Quantification Tool for Soil Properties Using Spectroscopic Data in R

Monitoring of CO2 Absorption Solvent in Natural Gas Process Using Fourier Transform Near-Infrared Spectrometry

Contact Info

Product

Resources

About

Monitoring of CO₂ Absorption Solvent in Natural Gas Process Using Fourier Transform Near-Infrared Spectrometry