A Relation-Augmented Fully Convolutional Network for Semantic Segmentation in Aerial Scenes

Abstract

Most current semantic segmentation approaches fall back on deep convolutional neural networks (CNNs). However, their use of convolution operations with local receptive fields causes failures in modeling contextual spatial relations. Prior works have sought to address this issue by using graphical models or spatial propagation modules in networks. But such models often fail to capture long-range spatial relationships between entities, which leads to spatially fragmented predictions. Moreover, recent works have demonstrated that channel-wise information also acts a pivotal part in CNNs. In this work, we introduce two simple yet effective network units, the spatial relation module and the channel relation module, to learn and reason about global relationships between any two spatial positions or feature maps, and then produce relation-augmented feature representations. The spatial and channel relation modules are general and extensible, and can be used in a plug-andplay fashion with the existing fully convolutional network (FCN) framework. We evaluate relation module-equipped networks on semantic segmentation tasks using two aerial image datasets, which fundamentally depend on long-range spatial relational reasoning. The networks achieve very competitive results, bringing signicant improvements over baselines.

Multi-Scale Context Aggregation for Semantic Segmentation of Remote Sensing Images

“…This module is inspired by the idea of non-local reasoning presented in [8,34,35]. The motivation is to model the long-range spatial correlations across different image sub-regions.…”

“…Precision = TP TP + FP , Recall = TP TP + FN (8) where TP, FP, and FN stand for true positive, false positive, and false negative, respectively. True positive denotes the number of correctly classified pixels, false positive denotes the number of pixels of other classes that are wrongly classified into a specific class, and false negative denotes the number of pixels of a given class that are wrongly classified into other categories.…”

“…Moreover, building on top of the HRNet, we propose a novel architecture that further enhances the aggregation of the context information. Contextual information is known to be crucial for the semantic segmentation of remote sensing images [8,9], since it offers important clues to the recognition of objects and regions. In the original design of the HRNet, the encoded features from four low-to-high resolution branches are concatenated together to produce the results.…”

AeroRIT: A New Scene for Hyperspectral Image Analysis

“…C Onvolutional neural networks (CNNs) are now being widely used for analyzing remote sensing imagery and though they have achieved some success, even the most well-designed CNNs for RGB imagery struggle to achieve a mean intersection-over-union of more than 80% on the ISPRS aerial datasets Vaihengen 1 and Potsdam 2 [1], [2]. This performance is in spite of the fact that these datasets have significantly higher spatial resolution with approximate ground sampling distances of 9cm (Vaihengen) and 5cm (Potsdam).…”

Extraction and Analysis of Blue Steel Roofs Information Based on CNN Using Gaofen-2 Imageries

“…For image recognition applications, the most important network structure in the deep learning algorithm is the CNN (Convolutional Neural Network) structure, which has the advantage of enabling computers to automatically extract feature information [19]. Many groups of researchers have begun to use CNN in many applications with impressive performance, such as image classification [20,21], object recognition [22,23], land use [24,25], and semantic segmentation [26,27].…”