3D Capsule Networks for Object Classification from 3D Model Data

Ahmad, Ayesha; Kakillioglu, Burak; Velipasalar, Senem

doi:10.1109/acssc.2018.8645256

Cited by 13 publications

(9 citation statements)

References 4 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Over the past few years, many modifications and extensions of the original ideas developed in [74] and [57] have been suggested [77][78][79][80][81] to further increase the robustness of capsule networks to image transformations and extend their application to more challenging domains, including image recognition under occlusion [81], geometric transformation occurring in visual SLAM and object detection in aerial surveillance [82,83], road sign recognition in autonomous driving systems [84], semantic scene segmentation [85] and action recognition [86]. The approaches have also been extended to 3D recognition tasks [87,88].…”

Section: Arbitrary Transformationsmentioning

confidence: 99%

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

Mumuni

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

One of the main challenges in machine vision relates to the problem of obtaining robust representation of visual features that remain unaffected by geometric transformations. This challenge arises naturally in many practical machine vision tasks. For example, in mobile robot applications like simultaneous localization and mapping (SLAM) and visual tracking, object shapes change depending on their orientation in the 3D world, camera proximity, viewpoint, or perspective. In addition, natural phenomena such as occlusion, deformation, and clutter can cause geometric appearance changes of the underlying objects, leading to geometric transformations of the resulting images. Recently, deep learning techniques have proven very successful in visual recognition tasks but they typically perform poorly with small data or when deployed in environments that deviate from training conditions. While convolutional neural networks (CNNs) have inherent representation power that provides a high degree of invariance to geometric image transformations, they are unable to satisfactorily handle nontrivial transformations. In view of this limitation, several techniques have been devised to extend CNNs to handle these situations. This article reviews some of the most promising approaches to extend CNN architectures to handle nontrivial geometric transformations. Key strengths and weaknesses, as well as the application domains of the various approaches are also highlighted. The review shows that although an adequate model for generalized geometric transformations has not yet been formulated, several techniques exist for solving specific problems. Using these methods, it is possible to develop task-oriented solutions to deal with nontrivial transformations.

show abstract

Section: Arbitrary Transformationsmentioning

confidence: 99%

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

Mumuni

2021

SN COMPUT. SCI.

View full text Add to dashboard Cite

show abstract

“…As the first method of using the capsule network architecture for point cloud edge detection, EDC-Net [24] is based on the extracted feature graph, by applying the attention module to the main capsule, and redesigning the loss function to obtain better results. Aiming at CNNs cannot sufficiently address the spatial relationship between features and require large amounts of data for training, the 3D point capsule networks [25] by Zhao and the method by Ahmad [26] were proposed. They extended the capsule network to the three-dimensional field by 3D convolution and verified its effectiveness through experiments.…”

Section: Motivation and Contributionmentioning

confidence: 99%

3D Mesh Model Classification with a Capsule Network

Zheng

Zhao

et al. 2021

Algorithms

View full text Add to dashboard Cite

With the widespread success of deep learning in the two-dimensional field, how to apply deep learning methods from two-dimensional to three-dimensional field has become a current research hotspot. Among them, the polygon mesh structure in the three-dimensional representation as a complex data structure provides an effective shape approximate representation for the three-dimensional object. Although the traditional method can extract the characteristics of the three-dimensional object through the graphical method, it cannot be applied to more complex objects. However, due to the complexity and irregularity of the mesh data, it is difficult to directly apply convolutional neural networks to 3D mesh data processing. Considering this problem, we propose a deep learning method based on a capsule network to effectively classify mesh data. We first design a polynomial convolution template. Through a sliding operation similar to a two-dimensional image convolution window, we directly sample on the grid surface, and use the window sampling surface as the minimum unit of calculation. Because a high-order polynomial can effectively represent a surface, we fit the approximate shape of the surface through the polynomial, use the polynomial parameter as the shape feature of the surface, and add the center point coordinates and normal vector of the surface as the pose feature of the surface. The feature is used as the feature vector of the surface. At the same time, to solve the problem of the introduction of a large number of pooling layers in traditional convolutional neural networks, the capsule network is introduced. For the problem of nonuniform size of the input grid model, the capsule network attitude parameter learning method is improved by sharing the weight of the attitude matrix. The amount of model parameters is reduced, and the training efficiency of the 3D mesh model is further improved. The experiment is compared with the traditional method and the latest two methods on the SHREC15 data set. Compared with the MeshNet and MeshCNN, the average recognition accuracy in the original test set is improved by 3.4% and 2.1%, and the average after fusion of features the accuracy reaches 93.8%. At the same time, under the premise of short training time, this method can also achieve considerable recognition results through experimental verification. The three-dimensional mesh classification method proposed in this paper combines the advantages of graphics and deep learning methods, and effectively improves the classification effect of 3D mesh model.

show abstract

“…Based on the concept of ''capsule'' that is introduced by Hinton et al [13] in 2010, Sabour et al [10] propose the dynamic routing algorithm between capsules and present a novel neural network named CapsNet. Since CapsNet can overcome some drawbacks of traditional CNNs, including the loss of location information in the pooling operation, and it performs well in many classic datasets, many researchers have paid attention to this [14], [15] and are devoted to improving its algorithm [16]- [20] or architecture [21]- [23].…”

Section: Related Workmentioning

confidence: 99%

RS-CapsNet: An Advanced Capsule Network

et al. 2020

View full text Add to dashboard Cite

Capsule Network is a novel and promising neural network in the field of deep learning, which has shown good performance in image classification by encoding features into capsules and constructing the part-whole relationships. However, the original Capsule Network is not suitable for the images with complex background due to its weak ability of feature extraction, a large number of training parameters, and the characteristic of explaining everything in the image. To address the above issues, we propose an advanced Capsule Network named RS-CapsNet, which uses Res2Net block to extract multi-scale features, and Squeeze-and-Excitation (SE) block to highlight useful features and suppress useless features. Meanwhile, we adopt the method of linear combination between capsules, which enhances the representation ability of capsules for detected object and reduces the number of capsules. Moreover, we propose the method of firstly constructing intermediate capsules that can represent most of the detected object, and then using the intermediate capsules and primary capsules to construct the classification capsules together. The experimental results show that proposed RS-CapsNet has better performance on CIFAR10, CIFAR100, SVHN, FashionMNIST, and AffNIST datasets, it can also provide better translation equivariance, and the number of training parameters is reduced by 65.11%.

show abstract

3D Capsule Networks for Object Classification from 3D Model Data

Cited by 13 publications

References 4 publications

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

CNN Architectures for Geometric Transformation-Invariant Feature Representation in Computer Vision: A Review

3D Mesh Model Classification with a Capsule Network

RS-CapsNet: An Advanced Capsule Network

Contact Info

Product

Resources

About