On the Time and Space Complexity of Genetic Programming for Evolving Boolean Conjunctions

Abstract

Genetic programming (GP) is a general purpose bio-inspired meta-heuristic for the evolution of computer programs. In contrast to the several successful applications, there is little understanding of the working principles behind GP. In this paper we present a performance analysis that sheds light on the behaviour of simple GP systems for evolving conjunctions of n variables (ANDn). The analysis of a random local search GP system with minimal terminal and function sets reveals the relationship between the number of iterations and the progress the GP makes toward finding the target function. Afterwards we consider a more realistic GP system equipped with a global mutation operator and prove that it can efficiently solve ANDn by producing programs of linear size that fit a training set to optimality and with high probability generalise well. Additionally, we consider more general problems which extend the terminal set with undesired variables or negated variables. In the presence of undesired variables, we prove that, if non-strict selection is used, then the algorithm fits the complete training set efficiently while the strict selection algorithm may fail with high probability unless the substitution operator is switched off. If negations are allowed, we show that while the algorithms fail to fit the complete training set, the constructed solutions generalise well. Finally, from a problem hardness perspective, we reveal the existence of small training sets that allow the evolution of the exact conjunctions even with access to negations or undesired variables.

Evolving boolean functions with conjunctions and disjunctions via genetic programming

“…With the limit on the tree size in place, our theoretical analysis reveals that the HVL-Prime mutation operator used in previous work [7,14], which either inserts, substitutes or deletes one node of the tree, may get stuck on local optima. Hence, the expected runtime of RLS-GP with the traditional HVL-Prime operator has in nite expected runtime.…”

“…∧x n . From [14], we repeat the observation that a conjunction of a distinct variables di ers from AND n on 2 n−a − 1 rows of the complete truth table.…”

“…Only recently, it has been proven that Boolean conjunctions of n variables can be evolved by RLS-GP [17] and (1+1) GP [14] algorithms in an expected polynomial number of iterations. e evolved conjunctions are exact when the complete truth table (i.e., the set of all 2 n possible inputs) is used to evaluate solution quality, and generalise well when tness is, more realistically, evaluated by sampling a polynomial number of inputs uniformly at random from the complete truth table in each iteration (i.e.…”

Computational Complexity Analysis of Genetic Programming

“…Theorem 23 has been extended to cover the (1 + 1) GP and (1 + 1) GP * algorithms, using a multiplicative drift theorem to provide a runtime bound on the expected time to fit a static polynomial-sized training set [25]. Additionally, a similar bound holds if, instead of a static training set, each iteration samples s independent rows of the complete truth table to compare the fitness of two solutions (using a dynamic training set).…”

“…Only recently, the time and space complexity of the (1 + 1) GP has been analyzed for evolving Boolean functions of arity n [29,25]. Solution quality was evaluated by comparing the output of the evolved programs with the target function on all possible inputs, or on a polynomially sized training set.…”