Transferability of the Deep Learning Mask R-CNN Model for Automated Mapping of Ice-Wedge Polygons in High-Resolution Satellite and UAV Images

Abstract

State-of-the-art deep learning technology has been successfully applied to relatively small selected areas of very high spatial resolution (0.15 and 0.25 m) optical aerial imagery acquired by a fixed-wing aircraft to automatically characterize ice-wedge polygons (IWPs) in the Arctic tundra. However, any mapping of IWPs at regional to continental scales requires images acquired on different sensor platforms (particularly satellite) and a refined understanding of the performance stability of the method across sensor platforms through reliable evaluation assessments. In this study, we examined the transferability of a deep learning Mask Region-Based Convolutional Neural Network (R-CNN) model for mapping IWPs in satellite remote sensing imagery (~0.5 m) covering 272 km2 and unmanned aerial vehicle (UAV) (0.02 m) imagery covering 0.32 km2. Multi-spectral images were obtained from the WorldView-2 satellite sensor and pan-sharpened to ~0.5 m, and a 20 mp CMOS sensor camera onboard a UAV, respectively. The training dataset included 25,489 and 6022 manually delineated IWPs from satellite and fixed-wing aircraft aerial imagery near the Arctic Coastal Plain, northern Alaska. Quantitative assessments showed that individual IWPs were correctly detected at up to 72% and 70%, and delineated at up to 73% and 68% F1 score accuracy levels for satellite and UAV images, respectively. Expert-based qualitative assessments showed that IWPs were correctly detected at good (40–60%) and excellent (80–100%) accuracy levels for satellite and UAV images, respectively, and delineated at excellent (80–100%) level for both images. We found that (1) regardless of spatial resolution and spectral bands, the deep learning Mask R-CNN model effectively mapped IWPs in both remote sensing satellite and UAV images; (2) the model achieved a better accuracy in detection with finer image resolution, such as UAV imagery, yet a better accuracy in delineation with coarser image resolution, such as satellite imagery; (3) increasing the number of training data with different resolutions between the training and actual application imagery does not necessarily result in better performance of the Mask R-CNN in IWPs mapping; (4) and overall, the model underestimates the total number of IWPs particularly in terms of disjoint/incomplete IWPs.

Understanding the Effects of Optimal Combination of Spectral Bands on Deep Learning Model Predictions: A Case Study Based on Permafrost Tundra Landform Mapping Using High Resolution Multispectral Satellite Imagery

“…In this study, we used a deep learning convolutional neural net (CNN) architecture called Mask-RCNN [ 42 ]. It is a semantic segmentation algorithm, which has been successfully applied in ice-wedge polygon mapping in Arctic regions [ 18 , 19 , 20 , 21 ]. The DL algorithm performs the object instance segmentation with outputs as predicted binary masks with classification information [ 18 , 19 , 20 , 21 ].…”

“…To the best of our knowledge, this is a poorly explored problem despite its validity in remote sensing applications. Here, we make an exploratory attempt to understand this problem based on a case study that branches out from our on-going project on Arctic permafrost thaw mapping from commercial satellite imagery [ 18 , 19 , 20 , 21 ].…”

Accuracy Assessment in Convolutional Neural Network-Based Deep Learning Remote Sensing Studies—Part 2: Recommendations and Best Practices

“…In developing training, validation, and testing datasets, it is also important to consider the impact of overlapping image chips. Many studies have noted reduced inference performance near the edge of image chips [67][68][69][70]; therefore, to improve the final map accuracy, it is common to generate image chips that overlap along their edges, and to use only the predictions from the center of each chip in the final, merged output. Overlapping chips are also sometimes generated to increase the number of samples available or represent objects of interest using different positions within the local scene [7,8,68,69].…”

A Novel Method for Automated Supraglacial Lake Mapping in Antarctica Using Sentinel-1 SAR Imagery and Deep Learning

“…CNNs are particularly suitable To overcome most of the described issues, we propose the use of a convolutional neural network (CNN) allowing to consider the spatial image context through convolutional extraction of feature maps from a given input image. CNNs are particularly suitable where traditional classification methods fail and were commonly used in computer vision and particularly for semantic segmentation and object detection tasks in remote sensing [31,32], e.g., for automated calving front detection using SAR and optical satellite imagery [33][34][35], for sea-land classification in optical satellite imagery [36], for multi-temporal crop type classification in optical Landsat imagery [37], for sea ice concentration estimation in dualpolarized RADARSAT-2 imagery [38], for automated mapping of ice-wedge polygons in high-resolution satellite and unmanned aerial vehicle images [39,40], or for building and road extraction in optical and aerial satellite imagery [41,42], to only name a few. In this study, we extract supraglacial lake extents from single-polarized Sentinel-1 imagery using a modified version of a CNN originally developed for semantic segmentation of biomedical images (U-Net) [43].…”

“…In Earth Observation, encoder-decoder designs and particularly variants of U-Net are most commonly used for semantic image segmentation due to their better performance compared to other approaches including naïve-decoder models [31]. With the main aim of this study being the derivation of lake areas by means of pixel-wise classifications, object detection approaches including multi-task instance segmentation (e.g., [39,40]) were not considered or tested. In detail, we aim at the segmentation of open water lakes as well as lakes that are roughened at their surface, e.g., by wind or return higher backscatter values due to a thin ice cover and appear with fuzzy edges, low contrast and speckle noise.…”