Population-based heuristic algorithms for continuous and mixed discrete-continuous optimization problems

Abstract

ii SummaryContinuous optimization problems are optimization problems where all variables have a domain that typically is a subset of the real numbers; mixed discretecontinuous optimization problems have additionally other types of variables, so that some variables are continuous and others are on an ordinal or categorical scale. Continuous and mixed discrete-continuous problems have a wide range of applications in disciplines such as computer science, mechanical or electrical engineering, economics and bioinformatics. These problems are also often hard to solve due to their inherent difficulties such as a large number of variables, many local optima or other factors making problems hard. Therefore, in this thesis our focus is on the design, engineering and configuration of high-performing heuristic optimization algorithms.We tackle continuous and mixed discrete-continuous optimization problems with two classes of population-based heuristic algorithms, ant colony optimization (ACO) algorithms and evolution strategies. In a nutshell, the main contributions of this thesis are that (i) we advance the design and engineering of ACO algorithms to algorithms that are competitive or superior to recent state-of-the-art algorithms for continuous and mixed discrete-continuous optimization problems, (ii) we improve upon a specific state-of-the-art evolution strategy, the covariance matrix adaptation evolution strategy (CMA-ES), and (iii) we extend CMA-ES to tackle mixed discrete-continuous optimization problems.More in detail, we propose a unified ant colony optimization (ACO) framework for continuous optimization (UACOR). This framework synthesizes algorithmic components of two ACO algorithms that have been proposed in the literature and an incremental ACO algorithm with local search for continuous optimization, which we have proposed during my doctoral research. The design of UACOR allows the usage of automatic algorithm configuration techniques to automatically derive new, high-performing ACO algorithms for continuous optimization. We also propose iCMAES-ILS, a hybrid algorithm that loosely couples IPOP-CMA-ES, a CMA-ES variant that uses a restart schema coupled with an increasing population iii size, and a new iterated local search (ILS) algorithm for continuous optimization. The hybrid algorithm consists of an initial competition phase, in which IPOP-CMA-ES and the ILS algorithm compete for further deployment during a second phase. A cooperative aspect of the hybrid algorithm is implemented in the form of some limited information exchange from IPOP-CMA-ES to the ILS algorithm during the initial phase. Experimental studies on recent benchmark functions suites show that UACOR and iCMAES-ILS are competitive or superior to other state-of-the-art algorithms.To tackle mixed discrete-continuous optimization problems, we extend ACO MV and propose CES MV , an ant colony optimization algorithm and a covariance matrix adaptation evolution strategy, respectively. In ACO MV and CES MV , the decision variables of an optimization prob...