Performance Evaluation of UAVSAR and Simulated NISAR Data for Crop/Noncrop Classification Over Stoneville, MS

Abstract

Timely and accurate large-scale data on agricultural activity are important for tracking and identifying management practices, cropland distribution, and for supporting food security programs. Needs of agricultural monitoring community involve a combination of high (<4 m) to moderate (<30 m) spatial resolutions, frequent revisit time, and open access, operational coverage for large spatial extents (Fritz et al., 2019). Because a majority of agricultural fields are just over 2 hectares in size, spatial resolutions should ideally be on the order of 100 m or less to adequately capture agricultural conditions and change (Yan & Roy, 2016). Revisit times on the order of weeks or less are desirable, because agricultural fields may undergo substantial change on diurnal or greater timescale due to processes such as tilling or precipitation (McNairn & Brisco, 2004). Collectively, since agriculture is closely tied to global markets and food security, there is a strong need for accurately monitoring agricultural activity at global scale (Becker-Reshef et al., 2019). Since retrievals made by Synthetic Aperture Radar (SAR) systems now meet or exceed these needs, they are well-suited for large-scale agricultural monitoring. Spaceborne SAR, such as the European Space Agency's (ESA) Sentinel-1 can map the Earth once every 6 to 12-days at moderate spatial resolution (Torres et al., 2012). SAR data provide valuable information useful for cropland identification, crop type classification and estimating yield (Betbeder et al., 2016; Huang et al., 2019; Whelen & Siqueira, 2018). For example, these data can be used to estimate biomass using backscatter magnitude, crop heights using interferometry and crop structure using polarimetry (Erten et al., 2016; Ferrazzoli et al., 1997; Wiseman et al., 2014). Also, unlike optical sensors, SAR can collect high quality data day and night and is largely unimpacted by atmospheric conditions, therefore having excellent potential for collecting dense time series. For these reasons, plus an increasing adoption of open data access policies, cheaper cloud computing resources, and a steady pipeline of future platforms, there has been a rapid increase in the use of SAR data sets with regards to agricultural applications and decision support systems.

Comparison between Dense L-Band and C-Band Synthetic Aperture Radar (SAR) Time Series for Crop Area Mapping over a NISAR Calibration-Validation Site

“…The Sentinel-based classifications performed close to or above 80% for a wide range of CV thr values, approximately between 0.3 to 0.65. This is nearly twice the range noted in a comparable study, but using L-band data over an agricultural site in Mississippi [16].…”

“…The Sentinel-based classifications performed close to or above 80% for a wide range of CVthr values, approximately between 0.3 to 0.65. This is nearly twice the range noted in a comparable study, but using L-band data over an agricultural site in Mississippi [16]. For interpreting how accuracy varies with respect to CVthr, it is important to point out that the ACI indicated a breakdown of crop and non-crop of 69% to 31%.…”

“…In the ROC optimization approach, the optimal CV thr value is that for which J was the largest. It is important to note that this is not necessarily where accuracy is the largest; but in practice, optimal CV thr values do closely match the best accuracy values [16]. The ROC curve is useful because it is an easily interpretable visual representation of classification performance.…”

“…The overall accuracy is calculated from the confusion matrix (Table 4) as: Solely relying on accuracy as a performance metric is problematic because of its inability to evaluate the model's classification performance. Trivial cases such as assigning all classifications to be crop (CV thr is 0) or non-crop (CV thr is large, e.g., 1 or greater), may still yield high accuracy values in cases where a study site consists entirely of crop or non-crop, or where a model indicates no skill [16]. Thus, we also employ Cohen's Kappa (κ) parameter to evaluate the model's performance.…”

“…Thus, we also employ Cohen's Kappa (κ) parameter to evaluate the model's performance. Unlike accuracy, κ also attempts to account for random chance using standard assumptions, and will usually indicate zero values-indicating results obtained by random chance-for the trivial cases above [16,34,35]. Following the methods of McHugh (2012) [35], we calculate Kappa in terms of the four confusion matrix categories (TP, TN, FP, FN) shown in Table 4:…”