MGFN: A Multi-Granularity Fusion Convolutional Neural Network for Remote Sensing Scene Classification

Abstract

Convolutional neural networks (CNNs) have been successfully used in remote sensing scene classification and identification due to their ability to capture deep spatial feature representations. However, the performance of deep models inevitably encounters a bottleneck when multimodality-dominated scene classification rather than single-modality-dominated scene classification is performed, due to the high similarity among different categories. In this study, we propose a novel multi-granularity fusion convolutional neural network (MGFN) to automatically capture the latent ontological features of remote sensing images. We firstly design a multigranularity module that can progressively crop input images to learn multigrained features, which can describe images to different degrees. Based on a comparison of different granularities, we then design a maxout-based module to learn the corresponding Gaussian covariance matrices of different granularities, which can extract second-order features to express the latent ontological essence of inputs and select the most distinguished inputs. We thirdly provide an adaptive fusion module to fuse all features via normalization to combine features of different degrees using the adaptive fused module. Finally, an SVM classifier is used to classify the fused matrix of every input image. Extensive experimentation and evaluations, particularly for multimodality-dominated scenes, demonstrate that the proposed network can achieve promising results for public remote sensing datasets.

Remote Sensing Scene Classification Via Multigranularity Alternating Feature Mining

“…[30] and [38] combined discriminative local features extraction technology with multi-branch feature fusion and attention algorithms to further gain competitive performance. Wang [38] and Zeng [39] cropped the input image to learn multi-grained regions, which are conducive to guide the network to learn more discriminative local detailed features, but the semantic correlation between different regions of the whole image is ignored. In order to solve the loss of local features and avoid increasing the complexity of network structure, we adopt a Region Confusion Mechanism (RCM) [26] to partition the input image into local patches and then splice together into image with smaller granularity levels of information.…”

Deep Learning Models Performance Evaluations for Remote Sensed Image Classification

“…The CNNs can be capable of spatial feature representations for RS image classifications using the convolution technique in the form of pixels [25]. This convolution process updates weights with each layer's provided non-linear activation function.…”