Evolution Strategies and Covariance Matrix Adaptation - Investigating New Shrinkage Techniques

Abstract

This paper discusses the covariance matrix adaptation in evolution strategies, a central and essential mechanism for the search process. Basing the estimation of the covariance matrix on small samples w.r.t. the search space dimension is known to be problematic. However, this situation is common in optimization raising the question, whether the performance of the evolutionary algorithms could be improved. In statistics, several approaches have been developed recently to improve the quality of the maximum-likelihood estimate. However, they are seldom applied in evolutionary computation. Here, we focus on linear shrinkage which requires relatively little additional effort. Several approaches and shrinkage targets are integrated into evolution strategies and analyzed in a series of experiments.