Denoising Autoencoders for Fast Combinatorial Black Box Optimization

Probst, Malte

doi:10.1145/2739482.2764691

Cited by 12 publications

(6 citation statements)

References 13 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, other neural network based EDAs such as DAE-EDA are computationally less expensive. Hence, they need considerably less time for solving the same benchmark instances to optimality ( [18,19,20]).…”

Section: Methodsmentioning

confidence: 99%

Deep Boltzmann Machines in Estimation of Distribution Algorithms for Combinatorial Optimization

Probst¹,

Rothlauf²

2015

Preprint

Self Cite

View full text Add to dashboard Cite

Estimation of Distribution Algorithms (EDAs) require flexible probability models that can be efficiently learned and sampled. Deep Boltzmann Machines (DBMs) are generative neural networks with these desired properties. We integrate a DBM into an EDA and evaluate the performance of this system in solving combinatorial optimization problems with a single objective. We compare the results to the Bayesian Optimization Algorithm. The performance of DBM-EDA was superior to BOA for difficult additively decomposable functions which are separable, i.e., concatenated deceptive traps of higher order. For most other benchmark problems, DBM-EDA cannot clearly outperform BOA, or other neural network-based EDAs. In particular, it often yields optimal solutions for a subset of the runs (with fewer evaluations than BOA), but is unable to provide reliable convergence to the global optimum competitively. At the same time, the model building process is fast, but computationally more expensive than that of other EDAs using probabilistic models from the neural network family, such as DAE-EDA.

show abstract

Section: Methodsmentioning

confidence: 99%

Deep Boltzmann Machines in Estimation of Distribution Algorithms for Combinatorial Optimization

Probst¹,

Rothlauf²

2015

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…On top of that, the NN can be applied in multiple ways. For instance, autoencoders [7,167], a particular class of NNs, have been used to sample new solutions in a way that resembles sampling in EDAs [121], but also to adaptively learn a "non-linear mutation operator" that attempts to widen the basins of attraction around the best individuals. In terms of the EDA performance, the potential of the neural model to capture intricate relationships among the variables is as relevant as the specific way it is utilized within the evolutionary algorithm.…”

Section: Neural and Deep Learning Models In Estimation Of Distributio...mentioning

confidence: 99%

“…The neural models that have been tried for EDAs, exhibit a variety of behaviors: autoencoders used in a traditional EDA scheme in [121] are extremely fast when compared with methods that learn Bayesian networks but they fail to achieve the same efficiency than BOA in terms of function evaluations. When used as a mutation distribution in [26] GA-dA (see Table 9) outperforms BOA in some problems (notably on the knapsack problem) but it is outperformed on the hierarchical HIFF function.…”

Section: Neural and Deep Learning Models In Estimation Of Distributio...mentioning

confidence: 99%

See 1 more Smart Citation

Gray-box optimization and factorized distribution algorithms: where two worlds collide

Santana

2017

Preprint

View full text Add to dashboard Cite

The concept of gray-box optimization, in juxtaposition to black-box optimization, revolves about the idea of exploiting the problem structure to implement more efficient evolutionary algorithms (EAs). Work on factorized distribution algorithms (FDAs), whose factorizations are directly derived from the problem structure, has also contributed to show how exploiting the problem structure produces important gains in the efficiency of EAs. In this paper we analyze the general question of using problem structure in EAs focusing on confronting work done in gray-box optimization with related research accomplished in FDAs. This contrasted analysis helps us to identify, in current studies on the use problem structure in EAs, two distinct analytical characterizations of how these algorithms work. Moreover, we claim that these two characterizations collide and compete at the time of providing a coherent framework to investigate this type of algorithms. To illustrate this claim, we present a contrasted analysis of formalisms, questions, and results produced in FDAs and gray-box optimization. Common underlying principles in the two approaches, which are usually overlooked, are identified and discussed. Besides, an extensive review of previous research related to different uses of the problem structure in EAs is presented. The paper also elaborates on some of the questions that arise when extending the use of problem structure in EAs, such as the question of evolvability, high cardinality of the variables and large definition sets, constrained and multi-objective problems, etc. Finally, emergent approaches that exploit neural models to capture the problem structure are covered.

show abstract

“…DAE-EDA uses a Denoising Autoencoder (DAE) as its probabilistic model [19,11,12]. The DAE is a feed-forward neural network, which is trained using the backpropagation algorithm.…”

Section: Algorithms For Comparisonmentioning

confidence: 99%

Generative Adversarial Networks in Estimation of Distribution Algorithms for Combinatorial Optimization

Probst

2015

Preprint

Self Cite

View full text Add to dashboard Cite

Estimation of Distribution Algorithms (EDAs) require flexible probability models that can be efficiently learned and sampled. Generative Adversarial Networks (GAN) are generative neural networks which can be trained to implicitly model the probability distribution of given data, and it is possible to sample this distribution. We integrate a GAN into an EDA and evaluate the performance of this system when solving combinatorial optimization problems with a single objective. We use several standard benchmark problems and compare the results to state-of-the-art multivariate EDAs. GAN-EDA doe not yield competitive results -the GAN lacks the ability to quickly learn a good approximation of the probability distribution. A key reason seems to be the large amount of noise present in the first EDA generations.

show abstract

Denoising Autoencoders for Fast Combinatorial Black Box Optimization

Cited by 12 publications

References 13 publications

Deep Boltzmann Machines in Estimation of Distribution Algorithms for Combinatorial Optimization

Deep Boltzmann Machines in Estimation of Distribution Algorithms for Combinatorial Optimization

Gray-box optimization and factorized distribution algorithms: where two worlds collide

Generative Adversarial Networks in Estimation of Distribution Algorithms for Combinatorial Optimization

Contact Info

Product

Resources

About