Can Differential Evolution Be an Efficient Engine to Optimize Neural Networks?

Baioletti, Marco; Bari, Gabriele Di; Poggioni, Valentina; Tracolli, Mirco

doi:10.1007/978-3-319-72926-8_33

Cited by 5 publications

(4 citation statements)

References 23 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…According to these new trends, neuroevolution has also been renewed [12][13][14][15]. The term of neuroevolution is used to identify the research area where evolutionary algorithms are used to construct and train artificial neural networks.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Differential Evolution for Neural Networks Optimization

et al. 2020

Self Cite

View full text Add to dashboard Cite

In this paper, a Neural Networks optimizer based on Self-adaptive Differential Evolution is presented. This optimizer applies mutation and crossover operators in a new way, taking into account the structure of the network according to a per layer strategy. Moreover, a new crossover called interm is proposed, and a new self-adaptive version of DE called MAB-ShaDE is suggested to reduce the number of parameters. The framework has been tested on some well-known classification problems and a comparative study on the various combinations of self-adaptive methods, mutation, and crossover operators available in literature is performed. Experimental results show that DENN reaches good performances in terms of accuracy, better than or at least comparable with those obtained by backpropagation.

show abstract

Section: Introductionmentioning

confidence: 99%

“…The system uses a direct encoding with a one-to-one mapping between the weights of the neural networks and values of individuals in the population. This system is an enhanced version of the system introduced in [12], where a preliminary implementation was described.…”

Section: Introductionmentioning

confidence: 99%

Differential Evolution for Neural Networks Optimization

et al. 2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…Baioletti et al [1] pick up the LE idea but replace the evolutionary algorithm with differential evolution (DE), which is a very successful optimizer for continuous parameter spaces [3]. The largest network they experiment with employs 7000 parameters.…”

Section: Related Workmentioning

confidence: 99%

Limited Evaluation Evolutionary Optimization of Large Neural Networks

Prellberg

Krämer

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Stochastic gradient descent is the most prevalent algorithm to train neural networks. However, other approaches such as evolutionary algorithms are also applicable to this task. Evolutionary algorithms bring unique trade-offs that are worth exploring, but computational demands have so far restricted exploration to small networks with few parameters. We implement an evolutionary algorithm that executes entirely on the GPU, which allows to efficiently batch-evaluate a whole population of networks. Within this framework, we explore the limited evaluation evolutionary algorithm for neural network training and find that its batch evaluation idea comes with a large accuracy trade-off. In further experiments, we explore crossover operators and find that unprincipled random uniform crossover performs extremely well. Finally, we train a network with 92k parameters on MNIST using an EA and achieve 97.6 % test accuracy compared to 98 % test accuracy on the same network trained with Adam. Code is available at https

show abstract

“…Given the conflicting nature of the above-mentioned objectives and motivated by the success of multi-objective evolutionary algorithms (MOEA) in other applications [10,11,12], we propose to model our method as a multi-objective optimization problem.…”

Section: Introduction and Related Workmentioning

confidence: 99%