No Free Lunch, Program Induction and Combinatorial Problems

Free lunches for neural network search

“…Among the many extension of NFL to a variety of domains, Woodward and Neil [28] have made some progress in assessing the applicability of NFL to program induction. In particular, they argued that there is a free lunch in a search space whenever there is a non-uniform many-to-one genotype-phenotype mapping, and that the mapping from syntax to behaviour in computer programs is one such mapping.…”

“…The reason why NFL would not normally be applicable to search in program spaces is that there are many more programs than behaviours 1 and that not all behaviours are equally likely. To formally show that, in general, not all behaviours are equally likely in program search spaces, Woodward and Neil [28] referred to Langdon's results on the limiting distribution of behaviour [12] and to the universal distribution [8]. However, there is a formal proof of the former result only for the case of register based machines, and no 1 This is valid if programs have a fixed number of inputs and outputs, each represented by a finite number of bits, and if we interpret a behaviour as a function from inputs to outputs.…”

Genetic Programming — Introduction, Applications, Theory and Open Issues

“…Among the many extension of NFL to a variety of domains, Woodward and Neil (2003) have made some progress in assessing the applicability of NFL to the search spaces explored by GP. In particular, they argued that there is a free lunch in a search space whenever there is a nonuniform many-to-one genotype-phenotype mapping, and that the mapping from syntax to functionality in GP is one such mapping.…”

“…Beyond this interesting counterexample, to show that in general not all functionalities are equally likely in program search spaces, Woodward and Neil (2003) referred to the results on the limiting distribution of functionality reviewed above and to the universal distribution (Kirchherr et al 1997) (informally this states that there are many more programs with a simple functionality than programs with a complex one). The latter result, however, applies to Turing complete languages, that is, to programs with memory and loops.…”

Free lunches for function and program induction

“…Among the many extensions of NFL to a variety of domains, Woodward and Neil [12] have made some progress in assessing the applicability of NFL to the search spaces explored by genetic programming (GP). In particular, they argued that there is a free lunch in a search space whenever there is a non-uniform many to one genotype-phenotype mapping, and that the mapping from syntax to functionality in GP is one such mapping.…”