Npaloss: Neighboring Pixel Affinity Loss for Semantic Segmentation in High-Resolution Aerial Imagery

Abstract

Abstract. The performance of semantic segmentation in high-resolution aerial imagery has been improved rapidly through the introduction of deep fully convolutional neural network (FCN). However, due to the complexity of object shapes and sizes, the labeling accuracy of small-sized objects and object boundaries still need to be improved. In this paper, we propose a neighboring pixel affinity loss (NPALoss) to improve the segmentation performance of these hard pixels. Specifically, we address the issues of how to determine the classifying difficulty of one pixel and how to get the suitable weight margin between well-classified pixels and hard pixels. Firstly, we convert the first problem into a problem that the pixel categories in the neighborhood are the same or different. Based on this idea, we build a neighboring pixel affinity map by counting the pixel-pair relationships for each pixel in the search region. Secondly, we investigate different weight transformation strategies for the affinity map to explore the suitable weight margin and avoid gradient overflow. The logarithm compression strategy is better than the normalization strategy, especially the common logarithm. Finally, combining the affinity map and logarithm compression strategy, we build NPALoss to adaptively assign different weights for each pixel. Comparative experiments are conducted on the ISPRS Vaihingen dataset and several commonly-used state-of-the-art networks. We demonstrate that our proposed approach can achieve promising results.

Multi‐scale network for remote sensing segmentation

“…Because remote sensing images usually contain multi‐scale and multi‐small targets. GoogLeNet [48] can effectively solve multi‐scale problems, but it is not deep enough for segmentation tasks. At the same time, the multi‐scale convolution kernel will bring a large number of parameters.…”

Urban Aquatic Scene Expansion for Semantic Segmentation in Cityscapes

“…Particularly, semantic segmentation [4,5] can serve as a foundational technology in applications ranging from smart city design to environmental monitoring. This process involves classifying each pixel in an image to demarcate distinct regions with semantic relevance, thus facilitating a detailed understanding of urban landscapes [6,7]. While the Cityscapes dataset [8] is a benchmark for assessing semantic segmentation algorithms in urban settings, it primarily focuses on terrestrial urban environments and lacks representation of aquatic regions [9,10].…”