Importance of Prediction Outlier Diagnostics in Determining a Successful Inter-Vendor Multivariate Calibration Model Transfer

Abstract

This paper reports on the transfer of calibration models between Fourier transform near-infrared (FT-NIR) instruments from four different manufacturers. The piecewise direct standardization (PDS) method is compared with the new hybrid calibration method known as prediction augmented classical least squares/partial least squares (PACLS/PLS). The success of a calibration transfer experiment is judged by prediction error and by the number of samples that are flagged as outliers that would not have been flagged as such if a complete recalibration were performed. Prediction results must be acceptable and the outlier diagnostics capabilities must be preserved for the transfer to be deemed successful. Previous studies have measured the success of a calibration transfer method by comparing only the prediction performance (e.g., the root mean square error of prediction, RMSEP). However, our study emphasizes the need to consider outlier detection performance as well. As our study illustrates, the RMSEP values for a calibration transfer can be within acceptable range; however, statistical analysis of the spectral residuals can show that differences in outlier performance can vary significantly between competing transfer methods. There was no statistically significant difference in the prediction error between the PDS and PACLS/PLS methods when the same subset sample selection method was used for both methods. However, the PACLS/PLS method was better at preserving the outlier detection capabilities and therefore was judged to have performed better than the PDS algorithm when transferring calibrations with the use of a subset of samples to define the transfer function. The method of sample subset selection was found to make a significant difference in the calibration transfer results using the PDS algorithm, while the transfer results were less sensitive to subset selection when the PACLS/PLS method was used.

Calibration Maintenance and Transfer Using Tikhonov Regularization Approaches

“…While not part of this paper, methods exist to determine oultiers 1,2 and indeed represent a critical aspect of calibration maintenance and transfer. 63 It has been shown that only a few samples are needed to update the model but these samples should be selected (if possible) based on a selection strategy that maximizes the ability of the standardization set to properly span the new conditions. With a standardization set properly spanning the new situation or secondary instrument, improved results should be obtainable compared to a random selection of samples for the standardization set.…”

Impact of standardization sample design on Tikhonov regularization variants for spectroscopic calibration maintenance and transfer

“…In this case, the standardization set should be representative of the new condition. However, the actual method used to select a standardization sample subset can affect the calibration transfer quality and, hence, prediction in the new conditions [11,37]. For example, the transformation quality of the piecewise direct standardization (PDS) algorithm was found to be sensitive to the method of subset selection, while prediction augmented classical least squares/partial least squares (PACLS/ PLS) was not [37].…”

“…However, the actual method used to select a standardization sample subset can affect the calibration transfer quality and, hence, prediction in the new conditions [11,37]. For example, the transformation quality of the piecewise direct standardization (PDS) algorithm was found to be sensitive to the method of subset selection, while prediction augmented classical least squares/partial least squares (PACLS/ PLS) was not [37]. It may be that a consensus (bagging, ensemble, fusion, stacking) model approach [38][39][40][41][42][43] would provide a better updated model compared to updating a model with one subset, i.e., multiple updated models are formed with different standardization sets and some form of composite prediction is reported from the models deemed acceptable.…”

Process analytical chemistry and chemometrics, Bruce Kowalski's legacy at The Dow Chemical Company

“…These methods make use of hybrid classical least squares (CLS)/PLS methods to affect the transfer but do require samples in common. Dow was involved in the study of this technology and brought to light an often overlooked aspect of calibration transfer, preservation of outlier diagnostics . It was found that when using piecewise direct standardization for this particular application, the transferred models incorrectly identified many of the prediction samples as outliers while still providing reasonable predictions.…”