Pyramidal RoR for image classification

Abstract

Abstract:The Residual Networks of Residual Networks (RoR) exhibits excellent performance in the image classification task, but sharply increasing the number of feature map channels makes the characteristic information transmission incoherent, which losses a certain of information related to classification prediction, limiting the classification performance. In this paper, a Pyramidal RoR network model is proposed by analysing the performance characteristics of RoR and combining with the PyramidNet. Firstly, based on RoR, the Pyramidal RoR network model with channels gradually increasing is designed. Secondly, we analysed the effect of different residual block structures on performance, and chosen the residual block structure which best favoured the classification performance. Finally, we add an important principle to further optimize Pyramidal RoR networks, drop-path is used to avoid over-fitting and save training time. In this paper, image classification experiments were performed on CIFAR-10/100 and SVHN datasets, and we achieved the current lowest classification error rates were 2.96%, 16.40% and 1.59%, respectively. Experiments show that the Pyramidal RoR network optimization method can improve the network performance for different data sets and effectively suppress the gradient disappearance problem in DCNN training.

Mixture separability loss in a deep convolutional network for image classification

“…The experiments showed that the classification performance of the proposed network achieved competitive performance on some public datasets, compared to conventional networks. It is worth mentioning that the classification accuracy can be further improved by applying the MSL to more advanced pyramidal residual networks [32, 33] which are currently state of the art on CIFAR‐10 (2.96%) and CIFAR‐100 (16.4%) datasets.…”

AutoSlim: Towards One-Shot Architecture Search for Channel Numbers

“…Other heuristics have also been explored. For example, the pyramidal rule [23,24] suggested to gradually increase the channels in all convolutions layer by layer, regardless of spatial size. Figure 1 visually summarizes these heuristics for setting channel numbers in a neural network.…”

“…Beyond the macro-level heuristics across entire network, recent works [6,13,24,25,26] have also digged into channel configuration for micro-level building blocks (a network building block is usually composed of several 1 × 1 and 3 × 3 convolutions). These micro-level heuristics have led to better speed-accuracy trade-offs.…”

“…Apart from these human-designed heuristics, efforts on automatically optimizing channel configuration have been made explicitly or implicitly. A recent work [27] suggested that many network pruning methods [1,28,29,30,31,32] [12,23,24], and inside network building blocks ((a) − (f )) [6,13,23,24,25,26].…”

“…Figure 1. Various heuristics for setting channel numbers across entire network ((A) − (B))[12,23,24], and inside network building blocks ((a) − (f ))[6,13,23,24,25,26].…”