Mask-R-FCN: A Deep Fusion Network for Semantic Segmentation

Abstract

Remote sensing image classification plays a significant role in urban applications, precision agriculture, water resource management. The task of classification in the field of remote sensing is to map raw images to semantic maps. Typically, fully convolutional network (FCN) is one of the most effective deep neural networks for semantic segmentation. However, small objects in remote sensing images can be easily overlooked and misclassified as the majority label, which is often the background of the image. Although many works have attempted to deal with this problem, making a trade-off between background semantics and edge details is still a problem. This is mainly because they are based on a single neural network model. To deal with this problem, a convolutional deep network with regions (R-CNN), which is highly effective for object detection is leveraged as a complementary component in our work. A learningbased and decision-level strategy is applied to fuse both semantic maps from a semantic model and an object detection model. The proposed network is referred to as Mask-R-FCN. Experimental results on real remote sensing images from the Zurich dataset, Gaofen Image Dataset (GID), and DataFountain2017 show that the proposed network can obtain higher accuracy than single deep neural networks and other machine learning algorithms. The proposed network achieved better average accuracies, which are approximately 2% higher than those of any other single deep neural networks on the Zurich, GID, and DataFoundation2017 datasets.

Deep Learning-Based Semantic Segmentation of Urban Features in Satellite Images: A Review and Meta-Analysis

“…Some papers intend multiple targets. The distribution is shown in Figure 2, which shows fewer studies on urban land cover [96,100,134,[136][137][138][139]. With the availability of VHR images, smaller urban features have been segmented in the majority of papers.…”

“…The most commonly used datasets are ISPRS Vaihingen 2D Semantic Labeling dataset [166] (38 papers) and ISPRS Potsdam 2D Semantic Labeling dataset [167] (27 papers) and IEEE GRSS Data Fusion Contest [168] (5 papers) collected their images using UAV. Other than UAV-based images, satellite images were obtained from RADARSAT-2 [136], Worldview-2 [102,134,156], Worldview-3 [149], Landsat-8 [96], SPOT [137,155], Gaofen [138,169], Quickbird, Sentinel-1 and 2 [147], Sentinel-2 and TerraSAR-X [135], Plan-etScope (Dove constellations) [164]. A particular research collected data from a plane [140].…”

“…Some modified the CE loss [142,163] and compared the use of CE with MFB and CE with focal loss function weighted by MFB [110]. Other loss functions include auxiliary loss [62,89], adversarial loss [127], regularized logistic regression [144], multinomial logistic loss [146,147,161], validation loss from Keras [121], softmax [104,106,134,138], bidirectional loss [76], moving average [169], negative log likelihood loss (NLLLoss) [164] and Dice coefficient and Tanimoto coefficient [108]. The distribution of loss function is shown in Figure 8.…”

“…Out of these, most (54 papers) compared to base models using modern DL architectures and 14 papers compared their results to traditional models. The traditional models include SVM [138], Extended Morphological Profiles (EMP) [156], conventional neural networks (NN) [152,158,159,170], Stochastic Expectation-Maximization (SEM) [136], Random Forest (RF) [55,102], Unary Potential Classifier [140], CRF, Simple Linear Iterative Clustering (SLIC) [137], maximum likelihood (ML) [134], k-nearest neighbor (kNN), Multi-layer Perceptron (MLP), multi-scale independent component analysis (MICA), stacked convolutional autoencoders (SCAE) [121], etc.…”

Feature Transformation Network for Few-Shot Learning

“…W ITH the development of deep neural network technology, no matter in target recognition [1]- [3], object detection [4], semantic segmentation [5], speech recognition [6], [7], or in text translation [8], these learning models based on deep neural networks have achieved significant progress. The success of these models depends on a large quantity of training samples.…”