Semantic Labeling in Remote Sensing Corpora Using Feature Fusion-Based Enhanced Global Convolutional Network with High-Resolution Representations and Depthwise Atrous Convolution

Abstract

One of the fundamental tasks in remote sensing is the semantic segmentation on the aerial and satellite images. It plays a vital role in applications, such as agriculture planning, map updates, route optimization, and navigation. The state-of-the-art model is the Enhanced Global Convolutional Network (GCN152-TL-A) from our previous work. It composes two main components: (i) the backbone network to extract features and ( i i ) the segmentation network to annotate labels. However, the accuracy can be further improved, since the deep learning network is not designed for recovering low-level features (e.g., river, low vegetation). In this paper, we aim to improve the semantic segmentation network in three aspects, designed explicitly for the remotely sensed domain. First, we propose to employ a modern backbone network called “High-Resolution Representation (HR)” to extract features with higher quality. It repeatedly fuses the representations generated by the high-to-low subnetworks with the restoration of the low-resolution representations to the same depth and level. Second, “Feature Fusion (FF)” is added to our network to capture low-level features (e.g., lines, dots, or gradient orientation). It fuses between the features from the backbone and the segmentation models, which helps to prevent the loss of these low-level features. Finally, “Depthwise Atrous Convolution (DA)” is introduced to refine the extracted features by using four multi-resolution layers in collaboration with a dilated convolution strategy. The experiment was conducted on three data sets: two private corpora from Landsat-8 satellite and one public benchmark from the “ISPRS Vaihingen” challenge. There are two baseline models: the Deep Encoder-Decoder Network (DCED) and our previous model. The results show that the proposed model significantly outperforms all baselines. It is the winner in all data sets and exceeds more than 90% of F 1 : 0.9114, 0.9362, and 0.9111 in two Landsat-8 and ISPRS Vaihingen data sets, respectively. Furthermore, it achieves an accuracy beyond 90% on almost all classes.

Semantic Segmentation of Aerial Imagery via Split-Attention Networks with Disentangled Nonlocal and Edge Supervision

“…We compared the performance of our model with multiple DCNN models recently published in academic journals (Table 6). The overall accuracy of our model exceeds most neural network models and is slightly lower than the two most accurate networks, ResUNet [6] and CASIA2 [29]. Our proposed method achieves the top three accuracies in the classes of building, car, low vegetation, and impervious, and only the accuracy of trees is mediocre.…”

“…In addition, residual modules are applied in multiple branches of the network to further correct the latent fitting residual caused by semantic gaps in multifeature fusion. Panboonyuen et al [29] presented a convolutional network, which is consists of a high-resolution representation (HRNet) [30] backbone, a set of feature fusion blocks, channel attention blocks and deconvolution blocks. In feature fusion, multilayer features of the HRNet are combined with features obtained by the global convolutional network (GCN) [31] to enhance local-global expression.…”

Building Change Detection Method to Support Register of Identified Changes on Buildings

“…Consequently, deep learning, especially the deep convolutional neural network (CNN), is a good and rewarding approach to automatic learning of object characteristics. Panboonyuen et al [36] proposed a new method to improve the accuracy of semantic segmentation. The proposed model showed superiority over other models on all tested data.…”

Automatic Identification and Dynamic Monitoring of Open-Pit Mines Based on Improved Mask R-CNN and Transfer Learning

“…When CNN performs instance segmentation, on account of the final feature map size being much smaller than the size of the input images, the final predicted split mask will be rough. However, the dilated convolution can control the rate of the convolution kernel and obtain different convolution receptive fields [35]. Therefore, dilated convolution solves the contradiction between improving the receptive fields and maintaining the size of the feature map in CNN.…”