MopIR: a machine-learning-oriented data preprocessing pipeline for precise analysis of infrared spectroscopy

Abstract

The application of infrared (IR) spectroscopy is limited due to intrinsically theoretical constraints. Machine learning offers a route for overcoming these limitations. Nevertheless, its efficacy significantly relies on the quality of data because irrelevant and/or redundant information hinders the success of learning the true signal. In this study, we introduced an intuitive multivariate method tailored for assessing the quality of spectra obtained from experimental replicates of identical samples. Additionally, we developed an analytical tool named "MopIR" (Machine-learning-Oriented Pipeline for InfraRed spectroscopy) designed to enhance the quality of IR spectra and therefore boost the performance in downstream machine learning tasks. The application of MopIR on simulated IR spectra yielded a substantial improvement in the quality of processed spectra. Furthermore, we extended the application of MopIR by combining it with machine learning algorithms on labeled IR spectra obtained from soil samples to predict carbon concentration within the samples. Our findings revealed a remarkable reduction in predictive errors and an enhancement in stability when comparing MopIR-processed data to raw data. In addition, we applied MopIR to IR spectra obtained from wood samples subjected to thermal-hydro-mechanical (THM) modifications, resulting in the reduction of background effect, signal amplification, and a boost in classification accuracy. Our results demonstrated an enhancement in machine learning tasks achieved through the improvement of data quality in the context of IR spectra analysis.