Covariance matrix repairing in Gaussian based EDAs

Abstract

Abstract-Gaussian models are widely adopted in continuous Estimation of Distribution Algorithms (EDAs). In this paper, we analyze continuous EDAs and show that they don't always work because of computation error: covariance matrix of Gaussian model can be ill-posed and Gaussian based EDAs using full covariance matrix will fail under specific conditions. It is a universal problem that all existing Gaussian based EDAs using full covariance matrix suffer from. Through theoretical analysis with examples of simulated data and experiments, we show that the ill-posed covariance matrix strongly affects those EDAs. This paper proposes a Covariance Matrix Repairing (CMR) method to fix ill-posed covariance matrix. CMR significantly improves the robustness of EDAs. Even some EDA's performance that was previously thought inefficient can be improved surprisingly with the help of CMR. CMR can also guarantee those EDAs to be used with small scale of population (but still should be large enough to find the global optimum) to accelerate the convergence rate while maintaining the quality of solutions.

Small Populations, High-Dimensional Spaces: Sparse Covariance Matrix Adaptation

“…Since evolutionary algorithms deploy selection based on rank or fitness, the assumption of the same distribution is not valid. This may be the reason as to why the literature research has resulted in only one previous approach [2]. There, the authors considered Gaussian based estimation of distribution algorithms.…”

“…[3]. While the approach in [2] resembles the Ledoit-Wolf estimator [3], it adapted the shrinkage intensity during the run.…”

Otimização em ambientes dinâmicos com variáveis contínuas empregando algoritmos de estimação de distribuição

“…Para o caso discreto, pode-se destacar: Baluja & Davies (1997), Pelikan & Mühlenbein (1999), Harik et al (2006), Shakya & McCall (2007), Gámez et al (2007), Santana et al (2010) e Li et al (2011). Já para o caso contínuo, pode-se citar: Sebag & Ducoulombier (1998), Gallagher et al (1999), Lu & Yao (2005), Dong & Yao (2007), Wierstra et al (2008), Dong & Yao (2008). Uma revisão sobre algoritmos de estimação de distribuição e uma compilação das propostas de algoritmos desenvolvidos até então podem ser encontradas em Pelikan et al (2002), Lozano (2006) e Armañanzas et al (2008).…”