A Multi-Branch Feature Fusion Strategy Based on an Attention Mechanism for Remote Sensing Image Scene Classification

Abstract

In recent years, with the rapid development of computer vision, increasing attention has been paid to remote sensing image scene classification. To improve the classification performance, many studies have increased the depth of convolutional neural networks (CNNs) and expanded the width of the network to extract more deep features, thereby increasing the complexity of the model. To solve this problem, in this paper, we propose a lightweight convolutional neural network based on attention-oriented multi-branch feature fusion (AMB-CNN) for remote sensing image scene classification. Firstly, we propose two convolution combination modules for feature extraction, through which the deep features of images can be fully extracted with multi convolution cooperation. Then, the weights of the feature are calculated, and the extracted deep features are sent to the attention mechanism for further feature extraction. Next, all of the extracted features are fused by multiple branches. Finally, depth separable convolution and asymmetric convolution are implemented to greatly reduce the number of parameters. The experimental results show that, compared with some state-of-the-art methods, the proposed method still has a great advantage in classification accuracy with very few parameters.

A Deformable Convolutional Neural Network with Spatial-Channel Attention for Remote Sensing Scene Classification

“…To demonstrate the superiority of our proposed method, it is compared with other methods on UCM, including Bidirectional adaptive feature fusion method (BDFF method) [25], Multiscale CNN (MCNN) [37], ResNet with weighted spatial pyramid matching collaborative representation-based classification (ResNet with WSPM-CRC) [38], VGG16 with multi-layer stacked covariance pooling (VGG16 with MSCP) [26], Gated bidirectional network (GBNet) [29], Feature aggregation CNN (FACNN) [39], Scale-free CNN (SF-CNN) [40], Deep discriminative representation learning with attention map method (DDRL-AM method) [41], and CNN based on attention-oriented multi-branch feature fusion (AMB-CNN) [42]. The training ratio of 80% is used on this dataset, and OA is taken as the evaluation index.…”

Remote Sensing Image Scene Classification Based on Global Self-Attention Module

“…In contrast, our global information extraction considers the linkage of various locations on the image, and the accuracy is 1.14% higher than GLANet when the training ratio is 10% and 0.55% higher than GLANet when the training ratio is 20%. [64] 91.03 ± 0.18 93.45 ± 0.17 SCCov [58] 89.30 ± 0.35 92.10 ± 0.25 DDRL-AM [40] 92.17 ± 0.08 92.46 ± 0.09 AMB-CNN [59] 88.99 ± 0.14 92.42 ± 0.14 ResNet-50+EAM [60] 90.87 ± 0.15 93.51 ± 0.12 Attention based Residual Network [61] -92.10 ± 0.30 ACNet [62] 91.09 ± 0.13 92.42 ± 0.16 Our Method 92.11 ± 0.06 94.00 ± 0.13…”

“…Compared with other methods, it can describe the content in the scene more effectively and has better accuracy. GLANet [64] 95.02 ± 0.28 96.66 ± 0.19 SCCov [58] 93.12 ± 0.25 96.10 ± 0.16 DDRL-AM [40] 92.36 ± 0.10 -AMB-CNN [59] 93.27 ± 0.22 95.54 ± 0.13 ResNet-50+EAM [60] 93.64 ± 0.25 96.62 ± 0.13 ACNet [62] 93. 33…”

Prototype Calibration with Feature Generation for Few-Shot Remote Sensing Image Scene Classification

“…Xie et al [43] developed a remote sensing image scene classification model with label augmentation, in which Kullback-Leibler divergence is utilized as the intra-class constraint to restrict the distribution of training data. Shi et al [44] proposed a lightweight CNN based on attention-oriented multi-branch feature fusion for remote sensing image scene classification.…”