Strategies for noise reduction and standardization of milk mid-infrared spectra from dairy cattle

Abstract

The use of Fourier-transform mid-infrared (FTIR) spectroscopy is of interest to the dairy industry worldwide for predicting milk composition and other novel traits that are difficult or expensive to measure directly. Although there are many valuable applications for FTIR spectra, noise from differences in spectral responses between instruments is problematic because it reduces prediction accuracy if ignored. The purpose of this study was to develop strategies to reduce the impact of noise and to compare methods for standardizing FTIR spectra in order to reduce between-instrument variability in multiple-instrument networks. Noise levels in bands of the infrared spectrum caused by the water content of milk were characterized, and a method for identifying and removing outliers was developed. Two standardization methods were assessed and compared: piecewise direct standardization (PDS), which related spectra on a primary instrument to spectra on 5 other (secondary) instruments using identical milkbased reference samples (n = 918) analyzed across the 6 instruments; and retroactive percentile standardization (RPS), whereby percentiles of observed spectra from routine milk test samples (n = 2,044,094) were used to map and exploit primary-and secondaryinstrument relationships. Different applications of each method were studied to determine the optimal way to implement each method across time. Industry-standard predictions of milk components from 2,044,094 spectra records were regressed against predictions from spectra before and after standardization using PDS or RPS. The PDS approach resulted in an overall decrease in root mean square error between industry-standard predictions and predictions from spectra from 0.190 to 0.071 g/100 mL for fat, from 0.129 to 0.055 g/100 mL for protein, and from 0.143 to 0.088 g/100 mL for lactose. Reductions in prediction error for RPS were similar but less consistent than those for PDS across time, but similar reductions were achieved when PDS coefficients were updated monthly and separate primary instruments were assigned for the North and South Islands of New Zealand. We demonstrated that the PDS approach is the most consistent method to reduce prediction errors across time. We also showed that the RPS approach is sensitive to shifts in milk composition but can be used to reduce prediction errors, provided that secondary-instrument spectra are standardized to a primary instrument with samples of broadly equivalent milk composition. Appropriate implementation of either of these approaches will improve the quality of predictions based on FTIR spectra for various downstream applications.

The evolving role of Fourier-transform mid-infrared spectroscopy in genetic improvement of dairy cattle

“…Many studies are based on spectra from a single instrument, and are therefore not required to account for the different variance-covariance structures of measurements from different instruments. In a study of spectra from 66 instruments, Grelet et al [42] showed considerable variability in the spectral responses of the instruments, while we have also observed that the distribution of MD values can be heterogeneous across instruments [43]. These results highlight the need to apply MD thresholds within instrument for the purpose of outlier removal.…”

“…More generally, Bittante and Cecchinato [44] showed that the transmittance of individual spectra wavenumbers had moderate to high heritability across most of the mid-infrared region and highlighted that absorbance peaks for non-water milk components were present in low signal-to-noise ratio regions and should be considered for investigation. The findings of these studies indicate that a prudent approach to removal of wavenumbers in low signal-to-noise ratio regions should be taken, retaining spectra from all regions in applications where the wavenumbers are considered independently, but removing them in applications where wavenumbers are considered in a multivariate manner [43].…”

“…A limitation of this approach is that it can only be used to adjust predictions for traits with pre-evaluated correction coefficients. More recent standardisation strategies have instead proposed calibrating the individual wavenumbers [42,43,48,49], allowing the correction of any trait predicted as a function of the spectral wavenumbers. Studies have shown that standardising individual wavenumbers can effectively reduce prediction errors when transferring calibration models between instruments for fat composition traits [42,48], as well as for calibration models for traits that are more difficult to predict reliably such as methane emissions and cheese yield [49].…”

“…Tiplady et al [43] showed that the most consistent standardisation approach for reducing prediction errors relies on analysing identical reference samples across all instruments, as outlined by Grelet et al [42]. Ideally, global reference sample sharing would be established, facilitating standardisation across instruments in different countries.…”

Symposium review: Big data, big predictions: Utilizing milk Fourier-transform infrared and genomics to improve hyperketonemia management

“…Furthermore, these proprietary predictions change over time, even within a manufacturer. Standardization efforts can reduce background noise and prediction errors and are sensitive to milk composition shifts (Tiplady et al, 2019). These differences may not be as evident to producers viewing results from a single DHI laboratory, but if research is based on separate platforms, there may be limitations to on-farm application based on the equipment used by the DHI or in-line system.…”