Deep network in network

Deep Residual Network in Network

“…However, changing the filter type is an important step to develop efficient CNNs. Using a nonlinear and more complex filter, such as an MLP filter, can generate more interesting results than using a simple linear filter [ 1 , 21 ]. Several architectures were based on this principle such as [ 1 , 21 , 42 ].…”

“…Using a nonlinear and more complex filter, such as an MLP filter, can generate more interesting results than using a simple linear filter [ 1 , 21 ]. Several architectures were based on this principle such as [ 1 , 21 , 42 ]. NIN [ 21 ] adopts a nonlinear filter: the multilayer perceptron (MLP) with a rectified linear unit (ReLU) used as an activation function.…”

“…NIN [ 21 ] adopts a nonlinear filter: the multilayer perceptron (MLP) with a rectified linear unit (ReLU) used as an activation function. In [ 1 ], DNIN model directly modifies NIN [ 21 ] in the sense of convolutional layer. It is represented in a three-layer stacking DMLPconv, which consists of two convolutional layers of size 3 × 3 and an eLU unit, used as an activation function instead of ReLU.…”

“…It is represented in a three-layer stacking DMLPconv, which consists of two convolutional layers of size 3 × 3 and an eLU unit, used as an activation function instead of ReLU. By incorporating micronetwork, DNIN [ 1 ] also increases depth. The depth of DNIN [ 1 ] is the same as that of NIN [ 21 ] and shares the same number of convolutional kernels.…”

“…They have had great success and reached the state of the art in several benchmarks. In this article, we address the degradation problem by introducing an efficient deep neural network architecture for computer vision, deep residual network in network, which takes its name from the deep network in the network article [ 1 ] in conjunction with the famous “deep residual learning for image recognition” [ 14 ]. The advantages of the architecture are experimentally verified on the CIFAR-10 classification challenges.…”