A benchmark set extension and comparative study for the HyFlex framework

Abstract

In this work we conduct a comparative study of several publicly available, state-of-the-art hyper-heuristics for HyFlex in order to assess their generality across domains. To this purpose we extend the HyFlex benchmark set with 3 new problem domains: The 0-1 Knap Sack, Quadratic Assignment and Max-Cut Problem. To our knowledge, this is the first public extension of the benchmark since the CHeSC 2011 competition. In addition, this is the first study testing the Fair-Share Iterated Local Search (FS-ILS) method, designed in prior research, using a semi-automated design approach, on new unseen problem domains. We show that, of the methods compared, Adap-HH (CHeSC 2011 winner) clearly perfoms the most consistently, overall. In addition, we identify a weakness of, as well as a way to further simplify the FS-ILS method. Finally, we found that, overall, the state-of-the-art methods compared, generalized much better than a naive baseline.

Performance of Selection Hyper-heuristics on the Extended HyFlex Domains

“…SSHH performs statistically significantly better than the remaining hyper-heuristics on QAP. The performance of the best hyper-heuristic from Table 1, SSHH is compared to the methods whose performances are reported in [1], including Adap-HH, which is the winner of the CHeSC 2011 competition [13], an Evolutionary Programming Hyper-heuristic (EPH) [12], Fair-Share Iterated Local Search with (FS-ILS) and without restart (NS-FS-ILS), Simple Random-All Moves (SR-AM) (denoted as AA-HH previously) and Simple Random-Improving or Equal (SR-IE) (denoted as ANW-HH previously). Table 2 summarises the results based on μ rank , μ norm , best and worst counts.…”

“…The ranks are then accumulated and averaged over all instances producing μ rank . -μ norm : the objective function values are normalised to values in the range [0,1] based on the following formula:…”

“…Initially, there were six minimisation problem domains implemented within HyFlex [14]. The HyFlex problem domains have been extended to include three more of them, including 0-1 Knapsack Problem (KP), Quadratic Assignment Problem (QAP) and MaxCut (MAC) [1]. In this study, we only consider the 'unseen' extended HyFlex problem domains to investigate the performance and the generality of some previously proposed well performing selection hyper-heuristics.…”

An Analysis of the Taguchi Method for Tuning a Memetic Algorithm with Reduced Computational Time Budget

“…Both parameter values vary in [0,1]. More details on the domain implementations, including low level heuristics and initialisation routines can be found on the competition website and in [1,15].…”

“…Six problem domains were implemented in the initial version of HyFlex: Maximum Satisfiability (MAX-SAT), One Dimensional Bin Packing (BP), Permutation Flow Shop (PFS), Personnel Scheduling (PS), Traveling Salesman (TSP) and Vehicle Routing (VRP). Three additional problem domains were added by Adriaensen et al [1] after the competition: 0-1 Knapsack (0-1 KP), Max-Cut, and Quadratic Assignment (QAP). Each domain contains a number of instances and problem specific components, including low level heuristics and an initialisation routine which can be used to produce an initial solution.…”