BIBED-Seg: Block-in-Block Edge Detection Network for Guiding Semantic Segmentation Task of High-Resolution Remote Sensing Images

Abstract

Edge optimization of semantic segmentation results is a challenging issue in remote sensing image processing. This article proposes a semantic segmentation model guided by a blockin-block edge detection network named BIBED-Seg. This is a twostage semantic segmentation model, where edges are extracted first and then segmented. We do two key works: The first work is edge detection, and we present BIBED, a block-in-block edge detection network, to extract the accurate boundary features. Here, the edge detection of multiscale feature fusion is first realized by creating the block-in-block residual network structure and devising the multilevel loss function. Second, we add the channel and spatial attention module into the residual structure to improve high-resolution remote sensing images' boundary positioning and detection accuracy by focusing on their channel and spatial dimensions. Finally, we evaluate our method on International Society for Photogrammetry and Remote Sensing (ISPRS) Potsdam and Vaihingen data sets and obtain ODS F-measure of 0.6671 and 0.7432, higher than other excellent edge detection methods. The second work is two-stage segmentation. First, the proposed BIBED is individually pretrained, and subsequently, the pretrained model is introduced into the entire segmentation network to extract boundary features. In the second segmentation stage, the edge detection network is used to constrain semantic segmentation results by loss cycles and feature bootstrapping. Our best model obtains the OA of 90.2%, 87.7%, and 81.5%, the IOU of 76.0%, 69.6%, and 61.3% on the ISPRS and WHDLD datasets, respectively.

Multielement-Feature-Based Hierarchical Context Integration Network for Remote Sensing Image Segmentation

“…Zheng et al [45] introduced an optimization algorithm based on Markov Random Fields (MRF) for multiscale edge-preserving in remote sensing segmentation. Sui et al [46] presented a segmentation network structure with block-wise edge detection. Unfortunately, the aforementioned methods mostly focus on the extraction of finer edges and overlook the importance of the fusion of edge information in the segmentation model for effective edge utilization.…”

Progressive Difference Amplification Network With Edge Sensitivity for Remote Sensing Image Change Detection

“…A high-quality CD map requires the smooth edge and a complete spatial structure. Edge cues can provide strong structural information of ground objects and are widely used in vision tasks, such as salient object detection [37], semantic segmentation [38], image compression [39], small target augmentation [40], etc. A sharp-edge detail highly advances global spatial structure, which is helpful in understanding the entire scene.…”

Deep-Learning-Based Semantic Segmentation of Remote Sensing Images: A Survey

“…Combining with boundary detection [106], [100], [105], [118], [123], [124], [125], [126], [127] Combining with change detection [128] Combining with super-resolution [129] Combining with region pixel segmentation [108], [130] Combining with target height prediction [131] Large-scale variation Multi-scale fusion Fuse features of multiple scales [110], [111], [132], [100], [133], [134] Foreground activation…”

“…Most of the existing methods [106], [100], [118], [123], [124] used a shared backbone network to extract features for two tasks at the same time, then designed a boundary-aware module on top of the backbone network to generate a boundary map, finally combined a segmentation loss and a boundary loss for model training. Furthermore, some studies [105], [125], [126] used a framework with two branches in a sequential or parallel manner to process two tasks. However, the dual-stream architecture faces the challenge of high model complexity.…”