Global Learnable Pooling With Enhancing Distinctive Feature for Image Classification

Zhang, Xiaohong

doi:10.1109/access.2020.2997078

Cited by 17 publications

(6 citation statements)

References 31 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…SVP outperforms conventional pooling methods such as GAP and GMP, and also it is more robust against noise. In addition, SVP is experimentally proven to provide further performance enhancement when it is applied to an existing knowledge distillation scheme such as [17]. On the other hand, SVP has a disadvantage that its computational complexity is somewhat higher than that of general pooling methods.…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Transformation of Non-Euclidean Space to Euclidean Space for Efficient Learning of Singular Vectors

Lee

Song

2020

IEEE Access

View full text Add to dashboard Cite

Singular value decomposition (SVD) is a popular technique to extract essential information by reducing the dimension of a feature set. SVD is able to analyze a vast matrix in spite of a relatively low computational cost. However, singular vectors produced by SVD have been seldom used in convolutional neural networks (CNNs). This is because the inherent properties of singular vectors such as sign ambiguity and manifold features make CNNs difficult to learn singular vectors. In order to overcome the limitations, this paper analyzes the undesirable properties of singular vectors and presents the transformation of singular vectors into Euclidean space as a smart solution. If the singular vectors are transformed to follow Euclidean geometry, SVD can be used for pooling to maintain the feature information well, which is called singular vector pooling (SVP). Since SVP can extract essential information from a feature map, it is robust against adversarial attacks in comparison to global average pooling. Thus, SVP shows a quantitative performance improvement of about 36% for the CIFAR10 dataset. In addition, we applied SVP to a knowledge distillation scheme that uses singular vectors in a restricted manner. As a result, SVP improved the performance by up to 1.7% for the CIFAR100 dataset.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Since global pooling was first introduced in [16], it has evolved in various ways, such as employing trainable variables [17] and adopting non-linear functions [18], [19]. On the other hand, a global pooling method using SVD was proposed [20], which was based on covariance pooling.…”

Section: Related Workmentioning

confidence: 99%

Transformation of Non-Euclidean Space to Euclidean Space for Efficient Learning of Singular Vectors

Lee

Song

2020

IEEE Access

View full text Add to dashboard Cite

show abstract

“…Convolutional Block Attention Module (CBAM) [ 131 ] contains two attention mechanisms: channel attention module followed by spatial attention module, where the channel attention module aims to capture the inter-channel relationship of features; on the other hand, the spatial attention module aims to capture the inter-spatial relationship of features. Global Learnable Pooling (GLPool) [ 147 ] can also be considered as an attention mechanism, where the weight of each pooling location is considered as a parameter and learned together with other network parameters in an end-to-end manner. [ 36 ] mathematically shows that the attention weighted pooling is equivalent to a low-rank approximation of second-order pooling.…”

Section: Pooling Methodsmentioning

confidence: 99%

Pooling in convolutional neural networks for medical image analysis: a survey and an empirical study

Nirthika

Manivannan

Ramanan

et al. 2022

Neural Comput & Applic

109

View full text Add to dashboard Cite

Convolutional neural networks (CNN) are widely used in computer vision and medical image analysis as the state-of-the-art technique. In CNN, pooling layers are included mainly for downsampling the feature maps by aggregating features from local regions. Pooling can help CNN to learn invariant features and reduce computational complexity. Although the max and the average pooling are the widely used ones, various other pooling techniques are also proposed for different purposes, which include techniques to reduce overfitting, to capture higher-order information such as correlation between features, to capture spatial or structural information, etc. As not all of these pooling techniques are well-explored for medical image analysis, this paper provides a comprehensive review of various pooling techniques proposed in the literature of computer vision and medical image analysis. In addition, an extensive set of experiments are conducted to compare a selected set of pooling techniques on two different medical image classification problems, namely HEp-2 cells and diabetic retinopathy image classification. Experiments suggest that the most appropriate pooling mechanism for a particular classification task is related to the scale of the class-specific features with respect to the image size. As this is the first work focusing on pooling techniques for the application of medical image analysis, we believe that this review and the comparative study will provide a guideline to the choice of pooling mechanisms for various medical image analysis tasks. In addition, by carefully choosing the pooling operations with the standard ResNet architecture, we show new state-of-the-art results on both HEp-2 cells and diabetic retinopathy image datasets.

show abstract

“…Global average pooling (GAP) is a global average of all pixels in the feature map of each channel and obtains the output of each feature map [39][40][41]. GAP directly removes the features of black box in the fully connected layer and gives each channel practical significance; then, the vectors composed of these output features will be sent to the classifier for classification directly [42]. Figure 6 shows the comparison between the fully connected layer and the global average pool layer.…”

Section: Global Averagementioning

confidence: 99%

A Novel Deep Convolutional Neural Network Based on ResNet‐18 and Transfer Learning for Detection of Wood Knot Defects

et al. 2021

View full text Add to dashboard Cite

Wood defects are quickly identified from an optical image based on deep learning methodology, which effectively improves wood utilization. Traditional neural network techniques have not yet been employed for wood defect detection due to long training time, low recognition accuracy, and nonautomatical extraction of defect image features. In this work, a model (so-called ReSENet-18) for wood knot defect detection that combined deep learning and transfer learning is proposed. The “squeeze-and-excitation” (SE) module is firstly embedded into the “residual basic block” structure for a “SE-Basic-Block” module construction. This model has the advantages of the features that are extracted in the channel dimension, and it is fused in multiscale with original features. Instantaneously, the fully connected layer is replaced with a global average pooling; consequently, the model parameters could be reduced effectively. The experimental results show that the accuracy has reached 99.02%, meanwhile the training time is also reduced. It shows that the proposed deep convolutional neural network based on ReSENet-18 combined with transfer learning can improve the accuracy of defect recognition and has a potential application in the detection of wood knot defects.

show abstract

Global Learnable Pooling With Enhancing Distinctive Feature for Image Classification

Cited by 17 publications

References 31 publications

Transformation of Non-Euclidean Space to Euclidean Space for Efficient Learning of Singular Vectors

Transformation of Non-Euclidean Space to Euclidean Space for Efficient Learning of Singular Vectors

Pooling in convolutional neural networks for medical image analysis: a survey and an empirical study

A Novel Deep Convolutional Neural Network Based on ResNet‐18 and Transfer Learning for Detection of Wood Knot Defects

Contact Info

Product

Resources

About