Empirical preprocessing methods and their impact on NIR calibrations: a simulation study

Abstract

The extraction of chemical information from dense particulate suspensions, such as industrial slurries and biological suspensions, using near-infrared (NIR) spectroscopic measurements is complicated by sample-to-sample path length variations due to light scattering. Empirical preprocessing techniques such as multiplicative scatter correction (MSC), extended MSC and derivatives have been applied to remove these effects and in some cases have shown promise. While the performance of these techniques and other related approaches is known to depend on the nature and extent of the variations and on the measurement configuration, detailed investigations into the efficacy of these approaches under various conditions have not been previously undertaken. The main obstacle to carrying out such investigations has been the lack of, and the difficulty in obtaining, an accurate and comprehensive experimental data set. In this work, simulations that generate 'actual' measurements were carried out to obtain 'experimental' spectroscopic data on particulate systems. This was achieved by solving the exact transport equation for light propagation. A model system comprising four chemical components with one consisting of spherical submicron particles was considered. Total diffuse transmittance and reflectance data generated through simulations for moderate particle concentrations were used as the basis for examining the effect of particle size variations and measurement configurations on the efficacy of a number of preprocessing techniques in enhancing the performance of partial least squares (PLS) models for predicting the concentration of one of the non-scattering chemical species. Additionally, a form of extended multiplicative signal correction based on considerations arising from fundamental light scattering theory is proposed and found to perform better than the other techniques for the cases considered in the study.

Effects of pre‐processing of Raman spectra on in vivo classification of nutrient status of microalgal cells

“…The superiority of EMSC compared with MSC presumably is because it corrects for wavelength dependent light scattering effects, enabling better differentiation between physical and chemical effects in the Raman spectra. The effectiveness of EMSC preprocessing compared with MSC or SNV for improving PLS modelling demonstrated here with Raman spectra supports recent simulation studies using near infrared spectroscopic data [12]. …”

Rapid Characterization and Prediction of Biomass Properties via Statistical Techniques

“…For example, multiplicative scatter correction averages the spectra first, and each individual spectrum is regressed by partial least squares to the total average. The details about the preprocessing methods are out of scope of this work, and can beregerred elsewhere to other papers [2,11].…”

Extraction of Chemical Information of Suspensions Using Radiative Transfer Theory to Remove Multiple Scattering Effects: Application to a Model Two-Component System

“…The EMSCL (Extended Multiplicative Scatter Correction with wavelength dependent log term) method which was found in [9] to be the best performing scatter correction technique for predicting the concentrations of non-scattering species, in the present case, needed fewer number of latent variables (LVs) and therefore is reported. When diffuse transmission measurements were used pre-processing with EMSCL provided a slight improvement.…”