Selective-candidate framework with similarity selection rule for evolutionary optimization

Zhang, Sheng Xin; Chan, Wing Shing; Peng, Zi Kang; Zheng, Shao Yong; Tang, Wallace K. S.

doi:10.1016/j.swevo.2020.100696

Cited by 20 publications

(11 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As reported in many published applications of DL in genomic predictions ( Bellot et al 2018 ; Abdollahi-Arpanahi et al 2020 ; Zingaretti et al 2020 ), we observed that retraining of a certain DL model with the same hyperparameter configuration and the same dataset produced slightly different predictions. This forced us to consider the variation in the predictive performance under the retraining in DE and post-DE model selection (see the methods section).…”

Section: Methodssupporting

confidence: 63%

“…For the real dataset, optimized MLPs fixed the nonlinear activation function “sigmoid.” We argue that the selected nonlinear activation reflects the increased complexity of polygenic inheritance in real datasets. Regarding this perspective, Zingaretti et al. (2020) indicated that in a real dataset, DL could model complex relationships by employing nonlinear functions, and they also observed that sigmoid-like hyperbolic tangent (“tanh”) was a safer choice overall.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

Heuristic hyperparameter optimization of deep learning models for genomic prediction

Han

Gondro

Reid

et al. 2021

G3 Genes|Genomes|Genetics

View full text Add to dashboard Cite

There is a growing interest among quantitative geneticists and animal breeders in the use of deep learning (DL) for genomic prediction. However, the performance of DL is affected by hyperparameters that are typically manually set by users. These hyperparameters do not simply specify the architecture of the model, they are also critical for the efficacy of the optimization and model fitting process. To date, most DL approaches used for genomic prediction have concentrated on identifying suitable hyperparameters by exploring discrete options from a subset of the hyperparameter space. Enlarging the hyperparameter optimization search space with continuous hyperparameters is a daunting combinatorial problem. To deal with this problem, we propose using differential evolution (DE) to perform an efficient search of arbitrarily complex hyperparameter spaces in DL models and we apply this to the specific case of genomic prediction of livestock phenotypes. This approach was evaluated on two pig and cattle datasets with real genotypes and simulated phenotypes (N = 7,539 animals and M = 48,541 markers) and one real dataset (N = 910 individuals and M = 28,916 markers). Hyperparameters were evaluated using cross validation. We compared the predictive performance of DL models using hyperparameters optimized by DE against DL models with “best practice” hyperparameters selected from published studies and baseline DL models with randomly specified hyperparameters. Optimized models using DE showed clear improvement in predictive performance across all three datasets. DE optimized hyperparameters also resulted in DL models with less overfitting and less variation in predictive performance over repeated retraining compared to non-optimized DL models.

show abstract

Section: Methodssupporting

confidence: 63%

Section: Methodsmentioning

confidence: 99%

Heuristic hyperparameter optimization of deep learning models for genomic prediction

Han

Gondro

Reid

et al. 2021

G3 Genes|Genomes|Genetics

View full text Add to dashboard Cite

show abstract

“…Kim and Lee (2019) reported that deep learning models with different hyperparameters could have the same predictive performance, which indicated that the best solution may not be unique. Zhang et al (2020) also indicated that superior solutions would prefer the closest candidates in evolutionary optimization algorithms. Therefore, we argue that DE evolves a population to where candidate solutions are increasingly similar to each other.…”

Section: Optimization Runtime Profilesmentioning

confidence: 99%

Heuristic hyperparameter optimization of deep learning models for genomic prediction

Han

Gondro

Reid

et al. 2020

Preprint

View full text Add to dashboard Cite

There is a growing interest among quantitative geneticists and animal breeders in the use of deep learning (DL) for genomic prediction. However, the performance of DL is affected by hyperparameters that are typically manually set by users. These hyperparameters do not simply specify the architecture of the model, they are also critical for the efficacy of the optimization and model fitting process. To date, most DL approaches used for genomic prediction have concentrated on identifying suitable hyperparameters by exploring discrete options from a subset of the hyperparameter space. Enlarging the hyperparameter optimization search space with continuous hyperparameters is a daunting combinatorial problem. To deal with this problem, we propose using differential evolution (DE) to perform an efficient search of arbitrarily complex hyperparameter spaces in DL models and we apply this to the specific case of genomic prediction of livestock phenotypes. This approach was evaluated on two pig and cattle datasets with real genotypes and simulated phenotypes (N=7,539 animals and M=48,541 markers) and one real dataset (N=910 individuals and M=28,916 markers). Hyperparameters were evaluated using cross validation. We compared the predictive performance of DL models using hyperparameters optimized by DE against DL models with “best practice” hyperparameters selected from published studies and baseline DL models with randomly specified hyperparameters. Optimized models using DE showed clear improvement in predictive performance across all three datasets.DE optimized hyperparameters also resulted in DL models with less overfitting and less variation in predictive performance over repeated retraining compared to non-optimized DL models.

show abstract

“…So, it is a crucial problem for this approach, how to design an effective individual selection mechanism to select the appropriate individual from the candidate pool. Aiming at the multi-strategy method and its important individual selection mechanism, we pay attention to an advanced DE optimization framework, namely, selective-candidate framework with similarity selection rule (SCSS), which was proposed by Zhang et al [28] in 2020. It consists of multiple candidates generation (MCG) and similarity selection (SS) rule.…”

Section: Nomenclature Descriptionsmentioning

confidence: 99%

A Multi-Strategy Differential Evolution Algorithm with Adaptive Similarity Selection Rule

Zheng,

Wen

2023

Symmetry

View full text Add to dashboard Cite

The differential evolution (DE) algorithm is a simple and efficient population-based evolutionary algorithm. In DE, the mutation strategy and the control parameter play important roles in performance enhancement. However, single strategy and fixed parameter are not universally applicable to problems and evolution stages with diverse characteristics; besides, the weakness of the advanced DE optimization framework, called selective-candidate framework with similarity selection rule (SCSS), is found by focusing on its single strategy and fixed parameter greedy degree (GD) setting. To address these problems, we mainly combine the multiple candidates generation with multi-strategy (MCG-MS) and the adaptive similarity selection (ASS) rule. On the one hand, in MCG-MS, two symmetrical mutation strategies, “DE/current-to-pbest-w/1” and designed “DE/current-to-cbest-w/1”, are utilized to build the multi-strategy to produce two candidate individuals, which prevents the over-approximation of the candidate in SCSS. On the other hand, the ASS rule provides the individual selection mechanism for multi-strategy to determine the offspring from two candidates, where parameter GD is designed to increase linearly with evolution to maintain diversity at the early evolution stage and accelerate convergence at the later evolution stage. Based on the advanced algorithm jSO, replacing its offspring generation strategy with the combination of MCG-MS and ASS rule, this paper proposes multi-strategy differential evolution algorithm with adaptive similarity selection rule (MSDE-ASS). It combines the advantages of two symmetric strategies and has an efficient individual selection mechanism without parameter adjustment. MSDE-ASS is verified under the Congress on Evolutionary Computation (CEC) 2017 competition test suite on real-parameter single-objective numerical optimization, and the results indicate that, of the 174 cases in total, it wins in 81 cases and loses in 30 cases, and it has the smallest performance ranking value, of 3.05. Therefore, MSDE-ASS stands out compared to the other state-of-the-art DEs.

show abstract

Selective-candidate framework with similarity selection rule for evolutionary optimization

Cited by 20 publications

References 56 publications

Heuristic hyperparameter optimization of deep learning models for genomic prediction

Heuristic hyperparameter optimization of deep learning models for genomic prediction

Heuristic hyperparameter optimization of deep learning models for genomic prediction

A Multi-Strategy Differential Evolution Algorithm with Adaptive Similarity Selection Rule

Contact Info

Product

Resources

About