Scene Recognition Using Deep Softpool Capsule Network Based on Residual Diverse Branch Block

Abstract

With the improvement of the quality and resolution of remote sensing (RS) images, scene recognition tasks have played an important role in the RS community. However, due to the special bird’s eye view image acquisition mode of imaging sensors, it is still challenging to construct a discriminate representation of diverse and complex scenes to improve RS image recognition performance. Capsule networks that can learn the spatial relationship between the features in an image has a good image classification performance. However, the original capsule network is not suitable for images with a complex background. To address the above issues, this paper proposes a novel end-to-end capsule network termed DS-CapsNet, in which a new multi-scale feature enhancement module and a new Caps-SoftPool method are advanced by aggregating the advantageous attributes of the residual convolution architecture, Diverse Branch Block (DBB), Squeeze and Excitation (SE) block, and the Caps-SoftPool method. By using the residual DBB, multiscale features can be extracted and fused to recover a semantic strong feature representation. By adopting SE, the informative features are emphasized, and the less salient features are weakened. The new Caps-SoftPool method can reduce the number of parameters that are needed in order to prevent an over-fitting problem. The novel DS-CapsNet achieves a competitive and promising performance for RS image recognition by using high-quality and robust capsule representation. The extensive experiments on two challenging datasets, AID and NWPU-RESISC45, demonstrate the robustness and superiority of the proposed DS-CapsNet in scene recognition tasks.

Fully automatic pipeline of convolutional neural networks and capsule networks to distinguish COVID-19 from community-acquired pneumonia via CT images

“…This is mainly because compared with the scalar value produced by the CNN models, the vector output by the capsule network can better represent the features. The vector formula used by the capsule network can offset the deficiency of the CNN and help the network represent the features in a strong and lightweight manner [ 67 ].…”

Improved YOLOv5-based image detection of cotton impurities

“…SoftPool is an improved pooling operation in the SPP module that preserves important detailed feature information, and the structure of SoftPool is shown in Figure 6, with the fused position in YOLOv5 shown in Figure 7. SoftPool uses Softmax 19 for weighted pooling, which keeps the expressiveness of the features and the operation is differentiable. With the ability to update the gradient for each backpropagation, SoftPool can combine every activation factor of the pooling kernel, increasing the memory footprint by only a fraction.…”

Deep Learning for Remote Sensing Image Scene Classification: A Review and Meta-Analysis

“…The filtered 50 articles target specific research problems, as depicted in Table 2, with a shared objective of enhancing remote sensing scene classification accuracy. Among the articles, 11 specifically focused on capturing more discriminative regions through the fusion of processed images in [75,78,83], multilayer fusion in [79,80,125,136,137], FC replaced by CapsNet [138] in [33,139], and pairwise comparison in [140]. To focus on key regions, attention mechanism is introduced in [100,103,104,106,107], while the use of classifier-detector is introduced in [86] and multiple instance learning (MIL) [141] in [142].…”