Generic parameter control with reinforcement learning

Abstract

Parameter control in Evolutionary Computing stands for an approach to parameter setting that changes the parameters of an Evolutionary Algorithm (EA) on-the-fly during the run. In this paper we address the issue of a generic and parameter-independent controller that can be readily plugged into an existing EA and offer performance improvements by varying the EA parameters during the problem solution process. Our approach is based on a careful study of Reinforcement Learning (RL) theory and the use of existing RL techniques. We present experiments using various state-of-the-art EAs solving different difficult problems. Results show that our RL control method has very good potential in improving the quality of the solution found without requiring additional resources or time and with minimal effort from the designer of the application.

Adaptive Parameter Selection in Evolutionary Algorithms by Reinforcement Learning with Dynamic Discretization of Parameter Range

“…In the Q-learning algorithm a single state is used and the ranges of parameter values are discretized a priori on five equally sized intervals as in the algorithm proposed by Karafotias et al The considered methods were tested on several real-valued functions with different landscapes and different number of local optima. We implemented the EARPC algorithm ourselves and we used the implementation of the method proposed by Karafotias et al kindly given by the authors of [7].…”

“…In the method proposed by Karafotias et al [7,8], a modification of the ε-greedy Q-learning algorithm is used. Let k denote the number of parameters being adjusted.…”

“…The pseudocode of the algorithm proposed by Karafotias et al is presented in Algorithm 2. We wrote this pseudocode based on the description of this algorithm given in [7].…”

“…Note that when we solve minimization problem with elitist selection, ft+1 is less than or equal to ft, so the reward is always positive. Parameters of reinforcement learning were taken from [7]. They are presented in Table 2.…”

Deep reinforcement learning based parameter control in differential evolution

“…Typical state features are fitness standard deviation, fitness improvement from parent to offspring, best fitness, and mean fitness [5,10]. Typical reward functions measure improvement achieved over the previous generation [10]. Other parameter control methods use an offline training phase to collect more data about the algorithm than what is available within a single run.…”