Gradient Based Evolution to Optimize the Structure of Convolutional Neural Networks

Mitschke, Norbert; Heizmann, Michael; Noffz, Klaus-Henning; Wittmann, Ralf

doi:10.1109/icip.2018.8451394

Cited by 14 publications

(2 citation statements)

References 7 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…[107]- [110], [112], [114], [115], [118], [120], [121], [123], [125], [126], [128]- [133], [135], [137], [141], [149], [156], [158], [161], [162], [172], [173], [175], [183], [192], [194]- [197] 1…”

Section: B Comparison On Cifar-10 and Cifar-100unclassified

A Survey on Evolutionary Neural Architecture Search

Liu,

Sun,

Xue

et al. 2020

Preprint

View full text Add to dashboard Cite

Deep Neural Networks (DNNs) have achieved great success in many applications, such as image classification, natural language processing and speech recognition. The architectures of DNNs have been proved to play a crucial role in its performance. However, designing architectures for different tasks is a difficult and time-consuming process of trial and error. Neural Architecture Search (NAS), which received great attention in recent years, can design the architecture automatically. Among different kinds of NAS methods, Evolutionary Computation (EC) based NAS methods have recently gained much attention and success. Unfortunately, there is not a comprehensive summary of the EC-based methods. This paper reviews 100+ papers of EC-based NAS methods in light of the common process. Four steps of the process have been covered in this paper including population initialization, population operators, evaluation and selection. Furthermore, current challenges and issues are also discussed to identify future research in this field.

show abstract

Section: B Comparison On Cifar-10 and Cifar-100unclassified

A Survey on Evolutionary Neural Architecture Search

Liu,

Sun,

Xue

et al. 2020

Preprint

View full text Add to dashboard Cite

show abstract

“…The existing methods usually employ random search [6], grid search [7], reinforcement learning [3], Bayesian optimization [8], evolutionary algorithms [9] and gradient-based methods [10] to explore the space of neural architectures. Although they give rise to a large number of studies for reporting more accurate classifiers, researchers are still faced with the challenge of computationally expensive simulations.…”

Section: Introductionmentioning

confidence: 99%

Neural Architecture Search for Time Series Classification

Rakhshani

Fawaz

Idoumghar

et al. 2020

2020 International Joint Conference on Neural Networks (IJCNN)

View full text Add to dashboard Cite

Neural architecture search (NAS) has achieved great success in different computer vision tasks such as object detection and image recognition. Moreover, deep learning models have millions or billions of parameters and applying NAS methods when considering a small amount of data is not trivial. Unlike computer vision tasks, labeling time series data for supervised learning is a laborious and expensive task that often requires expertise. Therefore, this paper proposes a simple-yet-effective fine-tuning method based on repeated k-fold cross-validation in order to train deep residual networks using only a small amount of time series data. The main idea is that each model fitted during cross-validation will transfer its weights to the subsequent folds over the rounds. We conducted extensive experiments on 85 instances from the UCR archive for Time Series Classification (TSC) to investigate the performance of the proposed approach. The experimental results reveal that our proposed model called NAS-T reaches new state-of-the-art TSC accuracy, by designing a single classifier that is able to beat HIVE-COTE: an ensemble of 37 individual classifiers.

show abstract