Validation of multivariate classification methods using analytical fingerprints – concept and case study on organic feed for laying hens

Abstract

Please cite this article as: Alewijn, Martin., van der Voet, Hilko., & van Ruth, Saskia., Validation of multivariate classification methods using analytical fingerprints − concept and case study on organic feed for laying hens.Journal of Food Composition and Analysis http://dx. 2Highlights  A validation procedure for multivariate classification methods is proposed  Combined validation of sample set composition, analytical method and performance  Class probability scores give more performance insight than classification results  Future correct classification rate derived from combined probability distributions  A case study on organic feed shows the new validation approach in practice AbstractMultivariate classification methods based on analytical fingerprints have found many applications in the food and feed area, but practical applications are still scarce due to a lack of a generally accepted validation procedure. This paper proposes a new approach for validation of this type of methods. A part of the validation procedure requires a description of qualitative aspects: the method's goal and purpose and adequateness of the sample sets used. The required quantitative performance is assessed from probabilistic data. Probability distributions are generalized using kernel density estimates, which allow meaningful interpolation and direct comparison and combination of different distributions. We propose inclusion of a permutation test, and provide suggestions for the assessment of the analytical repeatability in the method's probabilistic units. The latter can serve as a quality control measure. For assessment of the method's overall performance, we propose to apply the combined cross validation and external validation set probability distributions in order to obtain the best estimate for the method's performance on future samples. Qualitative and quantitative aspects are to be combined into a validation dossier stating performance for a well-defined purpose and scope. The proposed validation approach is applied to a case study: a binary classification discriminating organic from conventional laying hen feed based on fatty acid profiling that is essential to ensure the organic status of eggs for human consumption. For this case study, an expected accuracy for organic feed recognition of 96% is obtained for an explicitly defined scope.

The Sample, the Spectra and the Maths—The Critical Pillars in the Development of Robust and Sound Applications of Vibrational Spectroscopy

“…What appears to have improved in the last decades is the capability to manage the quality control, equation updates, and data analysis [105][106][107][108][109][110][111][112][113][114][115]122,123]. As mentioned above, in order to assess the accuracy of a calibration model and to avoid overfitting, validation procedures have to be applied; a calibration model without validation is nonsense [105][106][107][108][109][110][111][112][113][114][115]. Although in feasibility studies cross-validation can be the best practical method to demonstrate that a model can predict the measured property, the actual accuracy must be estimated with an appropriate test set or validation set [105][106][107][108][109][110][111][112][113][114][115].…”

“…For example, in leave-one-out cross-validation, one sample is removed from the dataset, and a calibration model is constructed for the remaining subset [104][105][106][107][108][109][110][111][112][113][114]. The removed samples are then utilised to calculate the prediction residual [105][106][107][108][109][110][111][112][113][114][115]. The process is repeated with other subsets until every sample has been left out once, and in the end, the variance of all prediction residuals is estimated.…”

Trends in Chemometrics: Food Authentication, Microbiology, and Effects of Processing

“…A group of authors studied the fatty acid profiles of 50 organic and 72 conventional chicken feed samples in the Netherlands (2009 to 2010) and used a classification method based on PLS‐DA to authenticate the farming system (Alewijn, van der Voet, & van Ruth, ). More than 92% of samples were correctly differentiated in the validation (100% sensitivity and 95% specificity in the training set) and the measurement of fatty acids coupled with PLS‐DA may represent a fast screening test to assess if organic laying‐hen feed is produced according to the organic protocol.…”

Tackling Fraudsters with Global Strategies to Expose Fraud in the Food Chain

“…Given the complexity of global pathways for the food supply chain, product fingerprinting in combination with chemometrics can be a useful tool for food fraud detection and control (Alewijn, van der Voet, & van Ruth, ). Consonni et al.…”