Algorithm of Stability-Analysis-Based Feature Selection for NIR Calibration Transfer

Abstract

For conventional near-infrared spectroscopy (NIR) technology, even within the same sample, the NIR spectral signal can vary significantly with variation of spectrometers and the spectral collection environment. In order to improve the applicability and application of NIR prediction models, effective calibration transfer is essential. In this study, a stability-analysis-based feature selection algorithm (SAFS) for NIR calibration transfer is proposed, which is used to extract effective spectral band information with high stability between the master and slave instruments during the calibration transfer process. The stability of the spectrum bands shared between the master and slave instruments is used as the evaluation index, and the genetic algorithm was used to select suitable thresholds to filter out the spectral feature information suitable for calibration transfer. The proposed SAFS algorithm was applied to two near-infrared datasets of corn oil content and larch wood density. Simultaneously, its calibration transfer performances were compared with two classical feature selection methods. The effects of different preprocessing algorithms and calibration transfer algorithms were also assessed. The model with the feature variables selected by the SAFS obtained the best prediction. The SAFS algorithm can simplify the spectral data to be transferred and improve the transfer efficiency, and the universality of the SAFS allows it to be used to optimize calibration transfer in various situations. By combining different preprocessing and classic feature selection methods with this, the sensitivity of the correlation between spectral data and component information are improved significantly, as well as the effect of calibration transfer, which will be deeply developed.

A Model Transfer Method among Spectrometers Based on Improved Deep Autoencoder for Concentration Determination of Heavy Metal Ions by UV-Vis Spectra

“…In recent years, with the rapid development of machine learning, more and more model transfer methods have been proposed by scholars. Zhang proposed a stability-analysis-based feature selection algorithm (SAFS) for near infrared (NIR) calibration transfer to tackle the variation of different spectrometers and environment change [ 17 ]. Chen proposed a new method that combines principal component analysis (PCA), weighted extreme value learning machine (ELM) and a TrAdaBoost algorithm [ 18 ].…”

Modeling of soluble solid content of PE‐packaged blueberries based on near‐infrared spectroscopy with back propagation neural network and partial least squares (BP–PLS) algorithm

“…Liu et al, 2021;J. Xue et al, 2019;Zhang et al, 2022). However, with the rapid development of NIR technology, there are already some portable NIR spectrometers as well as online NIR devices available for quick detection under industrial conditions (Porep et al, 2015).…”

“…The detection efficiency is significantly increased by the fact that the NIR spectrum is typically collected without sample preparation or the use of chemical reagents (Park et al., 2020). NIR detection technology has been widely used in agriculture, forestry, food, petroleum, chemical, pharmaceutical, and other research fields (C. Li, Li, et al., 2022; Q. Liu et al., 2021; J. Xue et al., 2019; Zhang et al., 2022). However, with the rapid development of NIR technology, there are already some portable NIR spectrometers as well as online NIR devices available for quick detection under industrial conditions (Porep et al., 2015).…”

Rapid Determination of Cellulose and Hemicellulose Contents in Corn Stover Using Near-Infrared Spectroscopy Combined with Wavelength Selection

“…With developments in science and technology, the acquisition accuracy of near-infrared spectrometers is increasing. If the correction model is established directly with the wavelength variables (WVs) of the whole spectrum, the accuracy and robustness of the model will eventually be affected, due to the weak correlation between some spectral WVs and the components [ 13 ]. To effectively extract the characteristic WVs (CWVs) with high correlation and to establish a simpler and more stable NIRS model, scholars have proposed using interval partial least squares [ 14 ], synergy partial least squares [ 15 ], backward partial least squares (BIPLS) [ 16 ], and other spectral area optimization algorithms, together with uninformative variable elimination [ 17 ], competitive adaptive weighted sampling (CARS) [ 18 ], and various other wavelength selection algorithms, and genetic algorithms (GA) [ 19 ], genetic simulated annealing algorithms (GSA) [ 20 ], ant colony algorithms [ 21 ], particle swarm optimization algorithms [ 22 ], and various other intelligent optimization algorithms to effectively filter out WVs.…”