A novel surrogate-assisted evolutionary algorithm applied to partition-based ensemble learning

Dushatskiy, Arkadiy; Alderliesten, Tanja; Bosman, Peter A. N.

doi:10.1145/3449639.3459306

Cited by 8 publications

(3 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Evolutionary algorithms are powerful general-purpose search algorithms and have also successfully been applied to NAS, see, e.g., [14,18]. Therefore, the next algorithm we consider is the parameterless version of the state-of-the-art EA Gene-pool Optimal Mixing Evolutionary Algorithm (P3-GOMEA) [7,8] (further referred to as GOMEA). P3-GOMEA is a model-based EA that attempts to detect and exploit linkage information during optimization.…”

Section: Search Algorithmsmentioning

confidence: 99%

“…P3-GOMEA is a model-based EA that attempts to detect and exploit linkage information during optimization. It was shown to perform better than other EAs on a range of problems [7], and, importantly, it does not have a population size hyperparameter which needs to be tuned in many other EAs. Further, we consider surrogate-assisted GOMEA (SAGOMEA): a modification of GOMEA which uses a surrogate model for cheap fitness estimation.…”

Section: Search Algorithmsmentioning

confidence: 99%

“…Further, we consider surrogate-assisted GOMEA (SAGOMEA): a modification of GOMEA which uses a surrogate model for cheap fitness estimation. SAGOMEA was designed specifically for discrete optimization problems with computationally expensive fitness functions and has been shown to be efficient [7]. Finally, we use a Bayesian optimization algorithm called Tree Parzen Estimator (TPE).…”

Section: Search Algorithmsmentioning

confidence: 99%

See 2 more Smart Citations

Heed the Noise in Performance Evaluations in Neural Architecture Search

Dushatskiy,

Alderliesten,

Bosman

2022

Preprint

Self Cite

View full text Add to dashboard Cite

Neural Architecture Search (NAS) has recently become a topic of great interest. However, there is a potentially impactful issue within NAS that remains largely unrecognized: noise. Due to stochastic factors in neural network initialization, training, and the chosen train/validation dataset split, the performance evaluation of a neural network architecture, which is often based on a single learning run, is also stochastic. This may have a particularly large impact if a dataset is small. We therefore propose to reduce the noise by having architecture evaluations comprise averaging of scores over multiple network training runs using different random seeds and cross-validation. We perform experiments for a combinatorial optimization formulation of NAS in which we vary noise reduction levels. We use the same computational budget for each noise level in terms of network training runs, i.e., we allow less architecture evaluations when averaging over more training runs. Multiple search algorithms are considered, including evolutionary algorithms which generally perform well for NAS. We use two publicly available datasets from the medical image segmentation domain where datasets are often limited and variability among samples is often high. Our results show that reducing noise in architecture evaluations enables finding better architectures by all considered search algorithms.

show abstract

Section: Search Algorithmsmentioning

confidence: 99%

Section: Search Algorithmsmentioning

confidence: 99%

Section: Search Algorithmsmentioning

confidence: 99%

See 1 more Smart Citation