Towards A Generic Computational Intelligence Library: Preventing Insanity

Metaheuristics "In the Large"

“…Algorithm descriptions in many papers on metaheuristics are far from precise enough to allow independent re-implementation, and public access to the associated source code is rarely mandated by editors or programme committees. As a consequence, replication studies are very uncommon: a recent such paper, the only replication study in metaheuristics of which the authors are aware, obtained results which were an order of magnitude worse than those originally claimed 2 As noted above, metaheuristics, and in particular Evolutionary Computation research, has developed something of a 'throw away' culture [78,23], in which a large percentage of researchers neither build upon the research implementations of their peers nor create such re-usable software artifacts themselves. This inability to consolidate is in contrast to other research areas that have successfully embraced re-use: for example the SBML standard 3 in systems biology, which allows the researcher to easily create test and deployment pipelines [57]; or the Taverna framework 4 used for workflow construction in a variety of other scientific disciplines.…”

“…In practice the extensibility of these frameworks is limited (though uniquely, to our knowledge, a progression towards mechanisms that enable wider reuse can be seen in CIlib [86,80,22,78]), and, crucially, implementation often requires the modification of internal source code, presenting a barrier to distribution, reuse, and understanding for other practitioners. We are left with a fragmented set of implementations that are incapable of representing an extensible design space for metaheuristics, without requiring modification to the frameworks themselves.…”

“…• Implementation: despite past efforts in developing software frameworks, there is a tendency to re-implement metaheuristics from scratch, hindering reproducibility and replicability [107,78,106].…”

“…This duplication of effort limits scientific progress; instead of building a cohesive body of knowledge consisting of robust scientific conclusions, accumulated wisdom in the field is more akin to "folklore": observations over individual algorithms and optimization problems, without a structured underlying narrative. In the words of Pamparà, there is a "throw away" research culture [78]: it is difficult to locate and compare against prior-art, and there is a lack of understanding as to which heuristics work best and in what context. Despite the availability of many software libraries, it is difficult to reuse existing implementations -in particular, it is extremely difficult to combine heuristics from different libraries or incorporate domain-specific knowledge in a general manner, which has hindered the creation of easily testable and deployable metaheuristic pipelines.…”

Extending the “Open-Closed Principle” to Automated Algorithm Configuration

“…Following initial work in Genetic Programming and exhaustive search (Briggs and O'Neill, 2008), monadic combinators were used in constraint programming (Schrijvers et al, 2013), with subsequent work in metaheuristics (Senington and Duke, 2013). The desirability of the monadic approach for metaheuristic design was recently advocated (Swan et al, 2015) as part of a wider research program, echoed by an emphatic call for re-usable libraries such as CILib (Pampara and Engelbrecht, 2015). The only work that we are aware of in which state-handling is used in explicit support of metaheuristic assembly is given by (Kocsis et al, 2015), who propose a combinator-based approach where component dependencies are amalgamated into a shared workspace.…”