Rank-based pooling for deep convolutional neural networks

Shi, Zenglin; Ye, Yangdong; Wu, Yunpeng

doi:10.1016/j.neunet.2016.07.003

Cited by 72 publications

(51 citation statements)

References 17 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In 2016, a new regime of pooling "rank based pooling" was proposed by Shi et al [45] to alleviate the scale problems which is confronted by value based pooling methods such as stochastic pooling. In RBP, a rank is assigned to all the activations in a pooling region, where the activations are sorted in descending order and rank are assigned in ascending order (start from one) ( Figure 10).…”

Section: Rank-based Pooling (Rbp)mentioning

confidence: 99%

See 1 more Smart Citation

Implications of Pooling Strategies in Convolutional Neural Networks: A Deep Insight

Sharma

Mehra

2019

Foundations of Computing and Decision Sciences

View full text Add to dashboard Cite

Convolutional neural networks (CNN) is a contemporary technique for computer vision applications, where pooling implies as an integral part of the deep CNN. Besides, pooling provides the ability to learn invariant features and also acts as a regularizer to further reduce the problem of overfitting. Additionally, the pooling techniques significantly reduce the computational cost and training time of networks which are equally important to consider. Here, the performances of pooling strategies on different datasets are analyzed and discussed qualitatively. This study presents a detailed review of the conventional and the latest strategies which would help in appraising the readers with the upsides and downsides of each strategy. Also, we have identified four fundamental factors namely network architecture, activation function, overlapping and regularization approaches which immensely affect the performance of pooling operations. It is believed that this work would help in extending the scope of understanding the significance of CNN along with pooling regimes for solving computer vision problems.

show abstract

Section: Rank-based Pooling (Rbp)mentioning

confidence: 99%

“…In RBP, a rank is assigned to all the activations in a pooling region, where the activations are sorted in descending order and rank are assigned in ascending order (start from one) ( Figure 10). Mathematically, rank assigning procedure can be represented as [45]:…”

Section: Rank-based Pooling (Rbp)mentioning

confidence: 99%

Implications of Pooling Strategies in Convolutional Neural Networks: A Deep Insight

Sharma

Mehra

2019

Foundations of Computing and Decision Sciences

View full text Add to dashboard Cite

show abstract

“…Intuitively, higher activation values are more important than lower ones, and thus the pooling weight should decrease as the ranking goes lower. However it may be sub-optimal to specify the weights manually as did in [9]. Hence we adopt a data-driven approach to learn these weights automatically.…”

Section: Top-k Poolingmentioning

confidence: 99%

Multi-Instance Multi-Scale CNN for Medical Image Classification

Liu

Sui

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Deep learning for medical image classification faces three major challenges: 1) the number of annotated medical images for training are usually small; 2) regions of interest (ROIs) are relatively small with unclear boundaries in the whole medical images, and may appear in arbitrary positions across the x, y (and also z in 3D images) dimensions. However often only labels of the whole images are annotated, and localized ROIs are unavailable; and 3) ROIs in medical images often appear in varying sizes (scales). We approach these three challenges with a Multi-Instance Multi-Scale (MIMS) CNN: 1) We propose a multi-scale convolutional layer, which extracts patterns of different receptive fields with a shared set of convolutional kernels, so that scale-invariant patterns are captured by this compact set of kernels. As this layer contains only a small number of parameters, training on small datasets becomes feasible; 2) We propose a "top-k pooling" to aggregate the feature maps in varying scales from multiple spatial dimensions, allowing the model to be trained using weak annotations within the multiple instance learning (MIL) framework. Our method is shown to perform well on three classification tasks involving two 3D and two 2D medical image datasets.

show abstract

“…In another recent study, a pooling method named as Rank-based pooling was developed [24]. This method has different variations in which activation values in each pooling region are first sorted according to their values.…”

Section: Related Workmentioning

confidence: 99%

Regp: A New Pooling Algorithm for Deep Convolutional Neural Networks

Yıldırım¹,

Baloğlu²

2019

NNW

View full text Add to dashboard Cite

In this paper, we propose a new pooling method for deep convolutional neural networks. Previously introduced pooling methods either have very simple assumptions or they depend on stochastic events. Different from those methods, RegP pooling intensely investigates the input data. The main idea of this approach is finding the most distinguishing parts in regions of the input by investigating neighborhood regions to construct the pooled representation. RegP pooling improves the efficiency of the learning process, which is clearly visible in the experimental results. Further, the proposed pooling method outperformed other widely used hand-crafted pooling methods on several benchmark datasets.

show abstract

Rank-based pooling for deep convolutional neural networks

Cited by 72 publications

References 17 publications

Implications of Pooling Strategies in Convolutional Neural Networks: A Deep Insight

Implications of Pooling Strategies in Convolutional Neural Networks: A Deep Insight

Multi-Instance Multi-Scale CNN for Medical Image Classification

Regp: A New Pooling Algorithm for Deep Convolutional Neural Networks

Contact Info

Product

Resources

About