Spatial dependence between training and test sets: another pitfall of classification accuracy assessment in remote sensing

Karasiak, Nicolas; Dejoux, Jean-François; Monteil, Claude; Sheeren, David

doi:10.1007/s10994-021-05972-1

Cited by 56 publications

(29 citation statements)

References 70 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, accuracy measures estimated in terms of proportion of area reveal low producer's accuracies, that is consistent given the strong influence (~95% of the map area) of the other LULC classes in the accuracy estimation. Furthermore, the models were validated using spatially independent samples (60 × 60 km spatial blocks) which decreased the classification accuracy (Karasiak et al., 2021) but provided more reliable results and raised new research questions (Braun, 2021). For example, the misclassification errors occurring in this study between dry (4030) and wet (4010/4020) heathland habitats (Fig.…”

Section: Discussionmentioning

confidence: 99%

“…Although this approach is interesting since it highlights advantages of the complementarity between environmental and vegetation variables, three issues remain: (i) its reproducibility at a national scale; (ii) the spatial resolution of the maps produced, which remains too coarse to detect the fine‐grain pattern of certain natural habitats such as heathlands; and (iii) the lack of using annual time series of satellite images for soil variables, which would better characterize the environment and discriminate among heathland habitats according to their phenology. Moreover, robust calibration of OCCs at a high spatial resolution and a national scale remains complex and notably requires hyperparameterization (Fernandez & Morales, 2019) based on spatially independent samples (Karasiak et al., 2021).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Large‐scale and fine‐grained mapping of heathland habitats using open‐source remote sensing data

Hubert‐Moy

Rozo

Perrin³

et al. 2022

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

Mapping natural habitats remains challenging, especially at a national scale. Although new open-access variables for vegetation and its environment and increased spatial resolution derived from satellite remote sensing data are available at the global scale, the relevance of these new variables for fine-grained mapping of natural habitats at a national scale remains underexplored. This study aimed to map the fine-grained pattern of four heathland habitats throughout France (550 000 km 2 ). Environmental (bioclimatic, soil and topographic) and spectral (vegetation) variables derived from MODerate resolution Imaging Spectroradiometer, Advanced Spaceborne Thermal Emission and Reflection Radiometer, and Sentinel-2 satellite data were analyzed using the MaxEnt classifier. Open-access field databases were used to calibrate and validate the classification, based on the threshold-independent area under the curve (AUC) index and the conventional F1-score. For each heathland habitat, potential and actual areas were mapped using environmental and spectral variables, respectively. The results showed high classification accuracy for potential (AUC 0.92-0.99) and actual (AUC 0.88-0.99) suitability maps of the four heathland habitats. Visual interpretation of maps of the probability of occurrence indicated that the fine-grained distribution of heathland habitat was detected satisfactorily. However, although the accuracy of the crisp map of combined classifications of actual heathland habitats was high (overall accuracy 0.72), estimated producer's accuracies in terms of proportion of area were low (<0.25). This study provides the first fine-grained pattern maps of heathland habitats at a national scale, thus highlighting the value of combining environmental and spectral variables derived from open-remote sensing data and open-source field databases. These suitability maps could support the identification of heathland habitats in the framework of national conservation policies.

show abstract

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Large‐scale and fine‐grained mapping of heathland habitats using open‐source remote sensing data

Hubert‐Moy

Rozo

Perrin³

et al. 2022

Remote Sens Ecol Conserv

View full text Add to dashboard Cite

show abstract

“…For the general model, the training data set contained 4210 individuals of 77 species and the test data set 1487 individuals of 72 species. Data for the 5 species missing from the test-set were collected only within plots selected for training, therefore no tree from these 5 species was suited or included in the held out testing data to minimize the effect of geographic autocorrelation on assessing accuracy (Karasiak, 2021). The resulting data represents 56% of the total tree species in the original unfiltered vegetation structure dataset and these species account for an average of 89% of individuals per site (Figure S.3).…”

Section: Discussionmentioning

confidence: 99%

Continental-scale hyperspectral tree species classification in the United States National Ecological Observatory Network

Marconi

Zou

Bohlman

et al. 2021

Preprint

View full text Add to dashboard Cite

Advances in remote sensing imagery and computer vision applications unlock the potential for developing algorithms to classify individual trees from remote sensing at unprecedented scales. However, most approaches to date focus on site-specific applications and a small number of taxonomic groups. This limitation makes it hard to evaluate whether these approaches generalize well across broader geographic areas and ecosystems. Leveraging field surveys and hyperspectral remote sensing data from the National Ecological Observatory Network (NEON), we developed a continental extent model for tree species classification that can be applied to the entire network including a wide range of US terrestrial ecosystems. We compared the performance of the generalized approach to models trained at each individual site, evaluating advantages and challenges posed by training species classifiers at the US scale. We evaluated the effect of geography, environmental, and ecological conditions on the accuracy and precision of species predictions. On average, the general model resulted in good overall classification accuracy (micro-F1 score), with better accuracy than site-specific classifiers (average individual tree level accuracy of 0.77 for the general model and 0.72 for site-specific models). Aggregating species to the genus-level increased accuracy to 0.83. Regions with more species exhibited lower classification accuracy. Trees were more likely to be confused with congeneric and co-occurring species and confusion was highest for trees with structural damage and in complex closed-canopy forests. The model produced accurate estimates of uncertainty, correctly identifying trees where confusion was likely. Using only data from NEON this single integrated classifier can make predictions for 20% of all tree species found in forest ecosystems across the US, suggesting the potential for broad scale general models for species classification from hyperspectral imaging.

show abstract

“…Selection. The quality of selected training samples not only has an important impact on the accuracy of remote sensing water body recognition [15,16]. In this paper, the following methods are designed, and the spectral characteristics of water bodies in the experiment are automatically selected.…”

Section: Training Samplementioning

confidence: 99%

Remote Sensing Image Water Body Recognition Algorithm Based on Deep Convolution Generating Network and Combined Features

Zhao

Feng

2022

Wireless Communications and Mobile Computing

View full text Add to dashboard Cite

The application of remote sensing images in water body recognition has become an effective method for ecological environment detection and evaluation, which has the disadvantages of low efficiency due to the existence of interpretation marks and rich interpretation experience in the current water body environment recognition, and overreliance on human experience. In this paper, the water body recognition method is applied to remote sensing images by combining the deep convolution generation network and the combined features, which has the advantage of high recognition accuracy. In the convolutional neural network, a five-layer convolutional neural network is used to construct a remote sensing water information extraction model, the transfer learning idea is introduced, and the densely connected feature fusion structure is added, so as to achieve the purposes of accelerating the convergence speed of the neural network, reducing the requirements of the neural network on the scale of training data, and reducing the loss of spatial hierarchical information and small object information. Compared with SVM, DBN, and CNN models, the experimental results show that the recognition accuracy of the proposed method is as high as 95. 69% under the constraint of scale window, which has a wide range of application scenarios and practical significance.

show abstract

Spatial dependence between training and test sets: another pitfall of classification accuracy assessment in remote sensing

Cited by 56 publications

References 70 publications

Large‐scale and fine‐grained mapping of heathland habitats using open‐source remote sensing data

Large‐scale and fine‐grained mapping of heathland habitats using open‐source remote sensing data

Continental-scale hyperspectral tree species classification in the United States National Ecological Observatory Network

Remote Sensing Image Water Body Recognition Algorithm Based on Deep Convolution Generating Network and Combined Features

Contact Info

Product

Resources

About