Spatial distribution as a key factor for evaluation of soil attributes prediction at field level using online near-infrared spectroscopy

Abstract

In soil science, near-infrared (NIR) spectra are being largely tested to acquire data directly in the field. Machine learning (ML) models using these spectra can be calibrated, adding only samples from one field or gathering different areas to augment the data inserted and enhance the models’ accuracy. Robustness assessment of prediction models usually rely on statistical metrics. However, how the spatial distribution of predicted soil attributes can be affected is still little explored, despite the fact that agriculture productive decisions depend on the spatial variability of these attributes. The objective of this study was to use online NIR spectra to predict soil attributes at field level, evaluating the statistical metrics and also the spatial distribution observed in prediction to compare a local prediction model with models that gathered samples from other areas. A total of 383 online NIR spectra were acquired in an experimental field to predict clay, sand, organic matter (OM), cation exchange capacity (CEC), potassium (K), calcium (Ca), and magnesium (Mg). To build ML calibrations, 72 soil spectra from the experimental field (local dataset) were gathered, with 59 samples from another area nearby, in the same geological region (geological dataset) and with this area nearby and more 60 samples from another area in a different region (global dataset). Principal components regression was performed using k-fold (k=10) cross-validation. Clay models reported similar errors of prediction, and although the local model presented a lower R2 (0.17), the spatial distribution of prediction proved that the models had similar performance. Although OM patterns were comparable between the three datasets, local prediction, with the lower R2 (0.75), was the best fitted. However, for secondary NIR response attributes, only CEC could be successfully predicted and only using local dataset, since the statistical metrics were compatible, but the geological and global models misrepresented the spatial patterns in the field. Agronomic plausibility of spatial distribution proved to be a key factor for the evaluation of soil attributes prediction at field level. Results suggest that local calibrations are the best recommendation for diffuse reflectance spectroscopy NIR prediction of soil attributes and that statistical metrics alone can mispresent the accuracy of prediction.

Soil Attributes Mapping with Online Near-Infrared Spectroscopy Requires Spatio-Temporal Local Calibrations

“…Only the details regarding the objective of this study are presented. Additional information and specificities can be found in the previous study that aimed to validate the methods and ML models used as the basis for the present study [28].…”

“…The strategy of using only local samples from the experimental area yielded the best predictions and, therefore, was applied in this study. Further description of ML models applied in this study can be found in the beforementioned paper [28]. These same models were applied to the online spectra acquired on day 21.…”

“…Researchers have identified the importance of local calibrations for predicting soil attributes using visible and NIR spectra [27][28][29], and although efforts are being made trying to overcome this limitation, there is not a well-established strategy [29][30][31]. Soil is a complex system of biological, physical, and chemical interactions that, at every moment, can reflect the energy-matter relationship upon which DRS is based.…”

“…Therefore, this study revisited an agricultural field where previous spectral acquisition and ML models were built and validated. The full report for the methods of acquisition, spectrophotometer functioning, and ML model validation for each soil attribute tested can be found in Canal Filho and Molin [28]. This study aimed to assess the performance of predictive models over time using the same sensor, operational, and instrumental conditions.…”

Online near-infrared spectroscopy for soil attributes prediction

“…This includes comprehending if the online NIR spectra is stable over the time, allowing repeatability of products generated. Researchers identified the importance of local calibrations for predicting soil attributes using visible and NIR spectra (Brown, 2007;Canal Filho and Molin, 2022;, and although efforts are being made trying to overcome this limitation, no definitive strategy is established (Gogé et al, 2014;Wetterlind et al, 2010). In this sense, it is also needed to assess the usefulness of ML calibrations based on DRS NIR spectra in terms of spatial and time specificity.…”