Length bias and search limitations in cartesian genetic programming

Abstract

In this paper we examine how Cartesian Genetic Programming's (CGP's) method for encoding directed acyclic graphs (DAGs) and its mutation operator bias the effective length of individuals as well as the distribution of inactive nodes in the genome. We investigate these biases experimentally using two CGP variants as comparisons: Reorder, a method for shuffling node ordering without effecting individual evaluation, and DAG, a method for removing the concept of node position. Experiments were performed on four problems tailored to highlight potential search limitations, with further testing on the 3-bit multiplier problem.Unlike previous work, our experiments show that CGP has an innate parsimony pressure that makes it very difficult to evolve individuals with a high percentage of active nodes. This bias is particularly prevalent as the length of an individual increases. Furthermore, these problems are compounded by CGP's positional biases which can make some problems effectively unsolvable. Both Reorder and DAG appear to avoid these problems and outperform Normal CGP on preliminary benchmark testing. Finally, these new techniques require more reasonable genome sizes than those suggested in current CGP, with some evidence that solutions are also more terse.

Cartesian Genetic Programming: Why No Bloat?

“…This effect is also seen in the increasing number of active nodes during evolution in Sections 5.1 and 5.2. This however is in keeping with the results found in [4] which investigated the effect of length bias in extreme cases where it was required that CGP used a very high percentage of active nodes; here the experiments were for real tasks typical of the applications of CGP.…”

“…Interestingly it is reported in [4] that CGP struggles to increase the number of active nodes during evolution even when a given task requires it; due to length bias. In Section 5.4 however it can be seen that CGP consistently used more active nodes on both tasks than for the neutral searches when given a low number of available nodes.…”

“…Length bias offers an alternative argument for why CGP does not suffer from bloat [4]. Length bias shows that nodes positioned closer to the chromosome inputs are much more likely to be active than those positioned nearer the outputs.…”

“…However, bloat does not appear in Cartesian Genetic Programming (CGP) [3]. In the literature there are two possible theories as to why CGP does not exhibit bloat; Neutral Genetic Drift (NGD) [3] and length bias [4]. This paper introduces both of these theories and then proceeds to empirically disprove them by removing the underlying assumptions each of them make.…”

Recurrent Cartesian Genetic Programming of Artificial Neural Networks

“…Overestimating the number of available nodes has shown to greatly aid evolution [45,63]; which is thought to heighten neutral genetic drift but could also be compensating for length bias [18,19]. The reason it is thought that such a simple evolutionary strategy is so effective for CGP is twofold.…”