MS-CapsNet: A Novel Multi-Scale Capsule Network

Xiang, Canqun; Zhang, Lu; Tang, Yi; Zou, Wenbin; Xu, Chen

doi:10.1109/lsp.2018.2873892

Cited by 210 publications

(124 citation statements)

References 10 publications

Supporting

Mentioning

122

Contrasting

Order By: Relevance

“…The CapsNet uses a group of neurons as a capsule to replace a neuron in the traditional neural network. In addition, the capsule is a vector to represent internal properties that can be used to learn part-whole relationships between various entities, such as objects or object parts, to achieve equivariance [36] and can solve the problem of traditional neural networks using fully connected layers cannot efficiently capture the hierarchical structure of the entities in images to preserve the spatial information [50].…”

Section: Introductionmentioning

confidence: 99%

Remote Sensing Image Scene Classification Using CNN-CapsNet

2019

View full text Add to dashboard Cite

Remote sensing image scene classification is one of the most challenging problems in understanding high-resolution remote sensing images. Deep learning techniques, especially the convolutional neural network (CNN), have improved the performance of remote sensing image scene classification due to the powerful perspective of feature learning and reasoning. However, several fully connected layers are always added to the end of CNN models, which is not efficient in capturing the hierarchical structure of the entities in the images and does not fully consider the spatial information that is important to classification. Fortunately, capsule network (CapsNet), which is a novel network architecture that uses a group of neurons as a capsule or vector to replace the neuron in the traditional neural network and can encode the properties and spatial information of features in an image to achieve equivariance, has become an active area in the classification field in the past two years. Motivated by this idea, this paper proposes an effective remote sensing image scene classification architecture named CNN-CapsNet to make full use of the merits of these two models: CNN and CapsNet. First, a CNN without fully connected layers is used as an initial feature maps extractor. In detail, a pretrained deep CNN model that was fully trained on the ImageNet dataset is selected as a feature extractor in this paper. Then, the initial feature maps are fed into a newly designed CapsNet to obtain the final classification result. The proposed architecture is extensively evaluated on three public challenging benchmark remote sensing image datasets: the UC Merced Land-Use dataset with 21 scene categories, AID dataset with 30 scene categories, and the NWPU-RESISC45 dataset with 45 challenging scene categories. The experimental results demonstrate that the proposed method can lead to a competitive classification performance compared with the state-of-the-art methods.

show abstract

Section: Introductionmentioning

confidence: 99%

Remote Sensing Image Scene Classification Using CNN-CapsNet

2019

View full text Add to dashboard Cite

show abstract

“…The final reconstructed structure consists of 512, 1024, and 784 channels in three fully connected layers. We also selected two derivatives of the standard capsule network: MS-CapsNet 12 and TextCaps. 13 Both of these derivatives were applied to the Fashion-MNIST and CIFAR-10 datasets, and the main improvements from both variants also lie in their feature extraction processes.…”

Section: Resultsmentioning

confidence: 99%

Feature and spatial relationship coding capsule network

et al. 2020

View full text Add to dashboard Cite

A capsule network encodes entity features into a capsule and maps a spatial relationship from the local feature to the overall feature by dynamic routing. This structure allows the capsule network to fully capture feature information but inevitably leads to a lack of spatial relationship guidance, sensitivity to noise features, and easy susceptibility to falling into local optimization. Therefore, we propose a novel capsule network based on feature and spatial relationship coding (FSc-CapsNet). Feature and spatial relationship extractors are introduced to capture features and spatial relationships, respectively. The feature extractor abstracts feature information from bottom to top, while attenuating interference from noise features, and the spatial relationship extractor provides spatial relationship guidance from top to bottom. Then, instead of dynamic routing, a feature and spatial relationship encoder is proposed to find the optimal combination of features and spatial relationships. The encoder abandons the idea of iterative optimization but adds the optimization process to the backpropagation. The experimental results show that, compared with the capsule network and its multiple derivatives, the proposed FSc-CapsNet achieves significantly better performance on both the Fashion-MNIST and CIFAR-10 datasets. In addition, compared with some mainstream deep learning frameworks, FSc-CapsNet performs quite competitively on Fashion-MNIST. © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.Traditional convolutional neural networks (CNNs) 1 have obvious limitations for exploring spatial relationships. The general method for classifying images of the same type taken from different angles is to train multiple neurons to process features and then add a top-level detection neuron to detect the classification results. This approach tends to remember the dataset rather than summarizing the solution, and it requires large amounts of training data to cover different variants and avoid overfitting. This characteristic also makes CNNs very vulnerable when dealing with tasks based on moved, rotated, or resized samples.Unlike CNNs, capsule networks (CapsuleNet) 2 use capsules 3 to capture a series of features and their variants. In the capsule network, higher-layer capsules are used to capture the overall features, such as "face" or "car," while the lower-layer capsules are used to capture local entity features such as "nose," "mouth," or "wheels," leading to a completely different approach than a convolutional network when abstracting the overall feature from local features. However, this is not enough. A complete identification process requires both bottom-up feature abstraction and top-down spatial relationship guidance. The capsule network defines a transformation matrix between adjacent capsule layers to implement feature abstraction. Then, dynamic rou...

show abstract

“…For low-level capsules, location information [20] is "position-coded" by the active capsule. As the hierarchy is raised, more and more positional information is "rate-coded" [21], [22] in the actual value component of the capsule output vector.…”

Section: Capsule Networkmentioning

confidence: 99%

CapsNet, CNN, FCN: Comparative Performance Evaluation for Image Classification

Jiang¹,

Wang²,

Liu³

et al. 2019

IJMLC

View full text Add to dashboard Cite

Image classification is one of the predominant tasks in computer vision. So far, there are many approaches in image classification, and the most typical methods are Convolutional Neural Networks (CNN), BOF-based algorithms, etc. Most of these methods have a good performance, but there are still some limitations. Capsule Network (CapsNet) is the most advanced algorithm, which realizes the operation based on active vector and dynamic routing, and can overcome limitations of the original algorithm. This paper attempts to apply CapsNet into image classification as well as another two efficient classification methods, which are CNN and Fully Convolutional Network (FCN). We use two datasets: MNIST and CIFAR-10 to train our model and tested the networks. Finally, compare and evaluate their performances in aspects of time cost, loss, accuracy and the number of parameters.

show abstract

MS-CapsNet: A Novel Multi-Scale Capsule Network

Cited by 210 publications

References 10 publications

Remote Sensing Image Scene Classification Using CNN-CapsNet

Remote Sensing Image Scene Classification Using CNN-CapsNet

Feature and spatial relationship coding capsule network

CapsNet, CNN, FCN: Comparative Performance Evaluation for Image Classification

Contact Info

Product

Resources

About