Monitoring organic carbon, total nitrogen, and pH for reclaimed soils using field reflectance spectroscopy

Sorenson, Preston; Small, Christina C.; Tappert, Michelle C.; Quideau, Sylvie A.; Drozdowski, Bonnie; Underwood, Andrew; Janz, Arnold

doi:10.1139/cjss-2016-0116

Cited by 49 publications

(38 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As a result, both MBL and Cubist were able to better predict soil physical and chemical properties. The results of this study are consistent with previous studies reporting their best model performance using local models or Cubist [25,87,88]. In particular, MBL models are able to better manage nonlinearity and extraneous information in the spectra by using spectrally similar neighbors in the reference sets to fit a new target function for each sample in the validation set [89].…”

Section: Best Model Performancesupporting

confidence: 89%

Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

et al. 2019

View full text Add to dashboard Cite

Diffuse reflectance spectroscopy (DRS) is emerging as a rapid and cost-effective alternative to routine laboratory analysis for many soil properties. However, it has primarily been applied in project-specific contexts. Here, we provide an assessment of DRS spectroscopy at the scale of the continental United States by utilizing the large (n > 50,000) USDA National Soil Survey Center mid-infrared spectral library and associated soil characterization database. We tested and optimized several advanced statistical approaches for providing routine predictions of numerous soil properties relevant to studying carbon cycling. On independent validation sets, the machine learning algorithms Cubist and memory-based learner (MBL) both outperformed random forest (RF) and partial least squares regressions (PLSR) and produced excellent overall models with a mean R2 of 0.92 (mean ratio of performance to deviation = 6.5) across all 10 soil properties. We found that the use of root-mean-square error (RMSE) was misleading for understanding the actual uncertainty about any particular prediction; therefore, we developed routines to assess the prediction uncertainty for all models except Cubist. The MBL models produced much more precise predictions compared with global PLSR and RF. Finally, we present several techniques that can be used to flag predictions of new samples that may not be reliable because their spectra fall outside of the calibration set.

show abstract

Section: Best Model Performancesupporting

confidence: 89%

Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

et al. 2019

View full text Add to dashboard Cite

show abstract

“…Since it is possible to repeatedly divide core, the number of ranges in surrds may be increased with each division. For example, as for the violation range shown in step (9) in Fig. 4, the method of range division is outlined in Fig.…”

Section: A Overviewmentioning

confidence: 99%

Formal Verification of a Decision-Tree Ensemble Model and Detection of Its Violation Ranges

Sato

Kuruma

Nakagawa

et al. 2020

IEICE Trans. Inf. & Syst.

View full text Add to dashboard Cite

As one type of machine-learning model, a "decisiontree ensemble model" (DTEM) is represented by a set of decision trees. A DTEM is mainly known to be valid for structured data; however, like other machine-learning models, it is difficult to train so that it returns the correct output value for any input value. Accordingly, when a DTEM is used in regard to a system that requires reliability, it is important to comprehensively detect input values that lead to malfunctions of a system (failures) during development and take appropriate measures. One conceivable solution is to install an input filter that controls the input to the DTEM, and to use separate software to process input values that may lead to failures. To develop the input filter, it is necessary to specify the filtering condition of the input value that leads to the malfunction of the system. Given that necessity, in this paper, we propose a method for formally verifying a DTEM and, according to the result of the verification, if an input value leading to a failure is found, extracting the range in which such an input value exists. The proposed method can comprehensively extract the range in which the input value leading to the failure exists; therefore, by creating an input filter based on that range, it is possible to prevent the failure occurring in the system. In this paper, the algorithm of the proposed method is described, and the results of a case study using a dataset of house prices are presented. On the basis of those results, the feasibility of the proposed method is demonstrated, and its scalability is evaluated.

show abstract

“…Since the 1000-2500 nm range (NIR and SWIR) is expected to have important spectral features associated with organic carbon [27,28] and this complete range is not covered by the Nano-Hyperspec imaging sensor, we used a spectroradiometer sensor FieldSpec 3 (Analytical Spectral Devices Inc.) to collect spectral data in the 1000-2500 nm region from the 111 samples. FieldSpec 3 has three detectors and covers a wider spectral range (350-2500 nm) with a band resolution (width) of 3 nm wide in the VNIR, and 10 nm in SWIR.…”

Section: Spectral Measurements and Analysismentioning

confidence: 99%

Ensemble Identification of Spectral Bands Related to Soil Organic Carbon Levels over an Agricultural Field in Southern Ontario, Canada

et al. 2019

View full text Add to dashboard Cite

The recent use of hyperspectral remote sensing imagery has introduced new opportunities for soil organic carbon (SOC) assessment and monitoring. These data enable monitoring of a wide variety of soil properties but pose important methodological challenges. Highly correlated hyperspectral spectral bands can affect the prediction and accuracy as well as the interpretability of the retrieval model. Therefore, the spectral dimension needs to be reduced through a selection of specific spectral bands or regions that are most helpful to describing SOC. This study evaluates the efficiency of visible near-infrared (VNIR) and shortwave near-infrared (SWIR) hyperspectral data to identify the most informative hyperspectral bands responding to SOC content in agricultural soils. Soil samples (111) were collected over an agricultural field in southern Ontario, Canada and analyzed against two hyperspectral datasets: An airborne Nano-Hyperspec imaging sensor with 270 bands (400–1000 nm) and a laboratory hyperspectral dataset (ASD FieldSpec 3) along the 1000–2500 nm range (NIR-SWIR). In parallel, a multimethod modeling approach consisting of random forest, support vector machine, and partial least squares regression models was used to conduct band selections and to assess the validity of the selected bands. The multimethod model resulted in a selection of optimal band or regions over the VNIR and SWIR sensitive to SOC and potentially for mapping. The bands that achieved the highest respective importance values were 711–715, 727, 986–998, and 433–435 nm regions (VNIR); and 2365–2373, 2481–2500, and 2198–2206 nm (NIR-SWIR). Some of these bands are in agreement with the absorption features of SOC reported in the literature, whereas others have not been reported before. Ultimately, the selection of optimal band and regions is of importance for quantification of agricultural SOC and would provide a new framework for creating optimized SOC-specific sensors.

show abstract

Monitoring organic carbon, total nitrogen, and pH for reclaimed soils using field reflectance spectroscopy

Cited by 49 publications

References 35 publications

Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

Accurate and Precise Prediction of Soil Properties from a Large Mid-Infrared Spectral Library

Formal Verification of a Decision-Tree Ensemble Model and Detection of Its Violation Ranges

Ensemble Identification of Spectral Bands Related to Soil Organic Carbon Levels over an Agricultural Field in Southern Ontario, Canada

Contact Info

Product

Resources

About