Spectral Encoder to Extract the Features of Near-Infrared Spectra for Multivariate Calibration

Abstract

An autoencoder architecture was adopted for near-infrared (NIR) spectral analysis by extracting the common features in the spectra. Three autoencoder-based networks with different purposes were constructed. First, a spectral encoder was established by training the network with a set of spectra as the input. The features of the spectra can be encoded by the nodes in the bottleneck layer, which in turn can be used to build a sparse and robust model. Second, taking the spectra of one instrument as the input and that of another instrument as the reference output, the common features in both spectra can be obtained in the bottleneck layer. Therefore, in the prediction step, the spectral features of the second can be predicted by taking the reverse of the decoder as the encoder. Furthermore, transfer learning was used to build the model for the spectra of more instruments by fine-tuning the trained network. NIR datasets of plant, wheat, and pharmaceutical tablets measured on multiple instruments were used to test the method. The multi-linear regression (MLR) model with the encoded features was found to have a similar or slightly better performance in prediction compared with the partial least-squares (PLS) model.

Infrared Spectral Analysis for Prediction of Functional Groups Based on Feature-Aggregated Deep Learning

“…As these methods are sensitive to the data, it is difficult for them to migrate between spectral data from different environments and instruments. 27 Deep learning (DL) is a subset of ML that develops "end-toend" predictive models through the self-learning of neural networks. A DL model can automatically extract features from the data, eliminating the need for human intervention in traditional machine learning methods and allowing the algorithm to be more general.…”

“…In many cases, however, it is necessary to evaluate a variety of data preprocessing methods for different applications, which increases the complexity of algorithms and the cost of experiments. As these methods are sensitive to the data, it is difficult for them to migrate between spectral data from different environments and instruments …”

Interpretable Perturbator for Variable Selection in near-Infrared Spectral Analysis

“…Deep learning (DL) is a neural network-based modeling method, which has been applied in the image processing, natural language processing, and spectral analysis. − Deep neural networks were proven powerful to learn critical patterns from raw spectra and improve the modeling due to the multiple processing layers. , However, the lack of interpretability of DL models is a hindrance to their application. To improve the interpretability of the models, a series of methods have been proposed, which can be divided into internal (interpretable models) and external algorithms.…”

Semi-Supervised Deep Learning-Based Multi-component Spectral Calibration Modeling for UV–vis and Near-Infrared Spectroscopy without Information Loss

“…Duan used an autoencoder architecture to extract the common features contained in the near infrared (NIR) spectral sets of calibration samples for the two instruments. The common features obtained at the bottleneck layer can be used to construct quantitative prediction models . Sun et al proposed an adaptively optimized gas analysis model to measure methane concentration with direct absorption spectroscopy.…”

“…The common features obtained at the bottleneck layer can be used to construct quantitative prediction models. 36 Sun et al proposed an adaptively optimized gas analysis model to measure methane concentration with direct absorption spectroscopy. The attention mechanism enhanced encoder-decoder structure composed of long short-term memory facilitates the automatic feature extraction and improves the robustness of the model under noise conditions.…”