Autonomous deep learning: A genetic DCNN designer for image classification

Ma, Benteng; Li, Xiang; Xia, Yong; Zhang, Yanning

doi:10.1016/j.neucom.2019.10.007

Cited by 94 publications

(74 citation statements)

References 37 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…On MNIST dataset, the proposed method achieves an accuracy of 99.54%, is superior to the deep CNN methods [3,4] that include a large number of convolutional layers. The test accuracy of SCNNB is similar to the state-of-the-art deep CNN [25] on MNIST. However, the [25] network consists of 5 × 5 convolution with 419/403 filters in the first / second convolutional layer and 7 × 7 convolution with 288 filters in the third convolutional layer respectively.…”

Section: Resultsmentioning

confidence: 69%

“…The test accuracy of SCNNB is similar to the state-of-the-art deep CNN [25] on MNIST. However, the [25] network consists of 5 × 5 convolution with 419/403 filters in the first / second convolutional layer and 7 × 7 convolution with 288 filters in the third convolutional layer respectively. Moreover, the SCNNB network has two 3 × 3 convolutions with 32 and 64 filters, respectively.…”

Section: Resultsmentioning

confidence: 69%

See 1 more Smart Citation

Shallow convolutional neural network for image classification

et al. 2019

View full text Add to dashboard Cite

Deep convolutional neural networks show great advantages in computer vision tasks, such as image classification and object detection. However, the networks have complex network structure which include a large number of layers such as convolutional layers and pooling layers. They greatly consume valuable computing and memory resources, and also hugely waste training time. Therefore, we propose a novel shallow convolutional neural network (SCNNB) to overcome the above limitations for image classification, which uses batch normalization techniques to accelerate training convergence and improve the accuracy. The SCNNB network has only 4 layers with small size of convolution kernels, which requires low time complexity and space complexity. In the experiments, we compare the SCNNB model with two variant models and the classical SCNN model on the two benchmark image datasets. Experimental results show that compared to SCNN model, the SCNNB model can quickly learn the features of the data and achieve the highest classification accuracy of 93.69% with 3.8 M time complexity on fashion-MNIST.

show abstract

Section: Resultsmentioning

confidence: 69%

Section: Resultsmentioning

confidence: 69%

Shallow convolutional neural network for image classification

et al. 2019

View full text Add to dashboard Cite

show abstract

“…On Fashion-MNIST dataset, our SCNND achieves 94.19% accuracy, which is lower than [5], [18], [19]. In [18], the network includes 7 convolutional layers with more filters. The convolutional layers of [5], [19] are 6 times more than the SCNND.…”

Section: Resultsmentioning

confidence: 99%

“…In Table 1, we can clearly see that the SCNND outperforms the methods of [3], [5], achieving high accuracy of 99.60% on MNIST dataset. Compared with the state-of-the-art method of [18] on MNIST, our SCNND includes two 3×3 convolutions with 32 and 64 filters, whereas the model of [18] uses the larger size of convolution (such as 7×7) and more filters (such as 419) to extract features.…”

Section: Experimental Parametersmentioning

confidence: 99%

Shallow Convolutional Neural Network for Image Recognition

Lei¹,

Liu²,

Jiang³

et al. 2019

IJCEE

View full text Add to dashboard Cite

Deep convolutional neural networks (DCNNs) have achieved state-of-the-art results for image recognition. However, these DCNNs with complex structure consist of many layers like convolutional layers, which require high time and computational complexity for training. Therefore, we propose a novel shallow convolutional neural network (SCNND) with dropout to address the problems of the DCNNs for image recognition. The SCNND with 4 layers can fast learning the features of the images, using dropout technology between two convolutional layers to improve recognition performance. Compared to the SCNNs, our SCNND includes 4 layers, with low time complexity and parameters. Experimental results show that our SCNND outperforms shallow CNN methods on Fashion-MNIST dataset.

show abstract

“…Sun ve arkadaşları[39] GA kullanarak elde ettikleri basit modeller ile karmaşık mimarilere AlexNet mimarisi[42] temel alınarak GA ile optimize edilen bir KSA önerilmiştir. Ma ve arkadaşları[43] tarafından önerilen çalışma bilinen resim tanıma problemleri ile test edildiğinde "state-of-the-art" mimarilere karşı başarılı sonuçlar vermiştir. Assunçao ve arkadaşları da[44] evrimsel algoritmalar ile otomatik olarak oluşturdukları KSA ile bilinen mimarilerle yarışabilir sonuçlar elde etmişlerdir.…”

unclassified

Konvolüsyonel Sinir Ağlarında Hiper-Parametre Optimizasyonu Yöntemlerinin İncelenmesi

Gülcü

Kuş

2019

Gazi Üniversitesi Fen Bilimleri Dergisi Part C: Tasarım Ve Teknoloji

View full text Add to dashboard Cite

In this study, we provide a review on the meta-heuristic methods like Genetic Algorithms, Particle Swarm Optimization, Differential Evolution and Bayes Optimization that have been used extensively to optimize hyper-parameters in Convolutional Neural Networks (CNN). We highlight the hyper-parameters that have been selected to be optimized in those studies along with the value domains of those parameters. These studies reveal that the number of layers, number of kernels and size of those kernels at each layer, learning rate and the batch size are among the hyper-parameters that affect the performance of the CNNs the most. Figure A. structure of convolutional neural networks Purpose: In this study, meta-heuristic methods that have been used to optimize convolutional neural networks are investigated. A performance comparison of these methods on different image datasets has been presented. The advantages and disadvantages of the optimization approaches have been presented with the aim of providing the user important points that should be considered during hyper-parameter selection process. Results: The definiton of "the best" set of hyper-parameters in convolutional neural networks depends on the problem or in this case, on the dataset. But it is clear from the studies that the selection of some parameters directly affect the performance of the networks. Number of layers, number of filters in each layer and size of each filter, regularization method, learning rate and batch size are among the most important parameters. It is easy to conclude that Genetic Algorithms (GA) are the most widely studied techniques used in hyper-parameter optimizaton. This is due to the fact that they yield successful results in most of the studies. While selecting the optimization method, one should consider the size of the problem, available computational budget and time. In addition, accuracy expectations should also be taken into account. For the problems with small hyper-parameter search space, methods like Grid Search would be sufficient, but for the problems with large search space, meta-heuristic methods would be more convenient. Conclusion: In this study, the effect of hyper-parameter optimization methods on classification performance is investigated. GA and Particle Swarm Optimization (PSO) methods are the two most-widely used meta-heuristics for hyper-parameter optimization. The computational burden of these methods can be justified with the accuracy improvement achieved with them. If the computational resources are limited, and it is desired to obtain good results in reasonable amount of time, then other methods like TPE and SMAC would be good choices.

show abstract

Autonomous deep learning: A genetic DCNN designer for image classification

Cited by 94 publications

References 37 publications

Shallow convolutional neural network for image classification

Shallow convolutional neural network for image classification

Shallow Convolutional Neural Network for Image Recognition

Konvolüsyonel Sinir Ağlarında Hiper-Parametre Optimizasyonu Yöntemlerinin İncelenmesi

Contact Info

Product

Resources

About