CroFuseNet: A Semantic Segmentation Network for Urban Impervious Surface Extraction Based on Cross Fusion of Optical and SAR Images

Abstract

The fusion of optical and synthetic aperture radar (SAR) images is a promising method to extract urban impervious surface (IS) accurately. Previous studies have shown that the feature-level fusion of optical and SAR images can significantly improve IS extraction. However, they generally use simple layer stacking for features fusion, ignoring the interaction between optical and SAR images. Besides, most of the features they used are shallow features manually extracted, such as texture and geometric features, lacking the use of high-level semantic features of images. The lack of publicly available IS datasets is considered as an obstacle that prevents the extensive use of deep learning models in IS extraction. Therefore, this study first creates an open and accurate IS dataset based on optical and SAR images, and then proposes a semantic segmentation network based on cross fusion of optical and SAR images features, namely CroFuseNet, for IS extraction. In CroFuseNet, we design a cross fusion module (CFM) to fuse features of optical and SAR images to achieve better complementarity between the two types of images. And we propose a multimodal features aggregation (MFA) module to aggregate specific high-level features from optical and SAR images. To validate the proposed CroFuseNet, we compare it with two classical machine learning algorithms and four state-of-theart deep learning models. The proposed model has the highest accuracy, with OA, MIoU and F1-Score of 97.77%, 0.9495 and 0.9770 respectively. The quantitative and qualitative experimental results demonstrate that the proposed model is superior to these comparative algorithms.

Automated Method for Artificial Impervious Surface Area Mapping in Temperate, Tropical, and Arid Environments Using Hyperlocal Training Data With Sentinel-2 Imagery

“…This result was similar to the accuracies found in our study; however, their F1-scores were higher than those in this study. Other recent studies have leveraged time series of SAR and multispectral imagery for impervious surface extent mapping with accuracies above 94%, often with various deep learning models [82][83][84][85]. One of the mentioned studies found SAR typically enhances overall accuracy by about 2%.…”

Learning SAR-Optical Cross Modal Features for Land Cover Classification

“…In contrast, synthetic aperture radar (SAR) photographs provide a rich array of landscape information from diverse material and physical viewpoints. SAR, an active sensor, detects backscatter information and is particularly sensitive to geometric features, such as surface roughness, temperature, and complex dielectric constants [3]. Although SAR images can penetrate obstacles and are impervious to water, they present challenges such as shortening, shadows, and speckle noise, making their imaging characteristics less intuitive for human visual interpretation.…”

Detecting Changes in Impervious Surfaces Using Multi-Sensor Satellite Imagery and Machine Learning Methodology in a Metropolitan Area

“…In many previous studies on urban impervious surfaces, multispectral data and SAR data have been fused together to extract urban targets, achieving improved classification results. However, previous research has mainly focused on the fusion of single optical data and SAR data, such as the fusion of Sentinel-2 MSI data with Sentinel-1 SAR data for impervious surface extraction [44,45], and the fusion of Landsat data with Sentinel-1 SAR data for impervious surface extraction [46]. Few studies have used multispectral data.…”