Evolvability from learning algorithms

Abstract

Valiant has recently introduced a framework for analyzing the capabilities and the limitations of the evolutionary process of random change guided by selection [24]. In his framework the process of acquiring a complex functionality is viewed as a substantially restricted form of PAC learning of an unknown function from a certain set of functions [23]. Valiant showed that classes of functions evolvable in his model are also learnable in the statistical query (SQ) model of Kearns [16] and asked whether the converse is true.We show that evolvability is equivalent to learnability by a restricted form of statistical queries. Based on this equivalence we prove that for any fixed distribution D over the instance space, every class of functions learnable by SQs over D is evolvable over D. Previously, only the evolvability of monotone conjunctions of Boolean variables over the uniform distribution was known [25]. On the other hand, we prove that the answer to Valiant's question is negative when distribution-independent evolvability is considered. To demonstrate this, we develop a technique for proving lower bounds on evolvability and use it to show that decision lists and linear threshold functions are not evolvable in a distribution-independent way. This is in contrast to distribution-independent learnability of decision lists and linear threshold functions in the statistical query model.

Distribution free evolvability of polynomial functions over all convex loss functions

“…Since the introduction of the evolvability model by L. Valiant in [11], significant work has been done to show both the power and the robustness to modeling variations of this computational framework for investigating how complexity can arise in a fixed environment [2,3,4,7,10]. In this work we present two complementary constructions, which extend this body of work in a new and very natural direction: while previous papers studied evolvability of Boolean functions (from {0, 1} n → {−1, 1}) we here consider functions from R n → R m .…”

“…For the original Boolean framework of evolvability, Feldman showed that, as long as the underlying distribution is known, then the class SQ (statistical queries) defined by Kearns in [8] exactly characterizes the classes of evolvable functions [2]. SQ is both a powerful and natural framework, and seems to capture most of the power of PAC learning.…”

PAC learning and genetic programming

“…The most classical approach in computational learning theory is to study this within the approximately correct (PAC) learning framework of Valiant [14]. Recently, it has been shown that a large class of functions is evolvable, i. e. PAC learnable by an evolutionary algorithm [15,3,4]. However, the goal of these papers is to understand natural evolution from a theoretical point of view rather than giving explanations why and how evolutionary algorithms that are used in practice work.…”

Fixed-Parameter Evolutionary Algorithms and the Vertex Cover Problem

“…A comprehensive presentation of the different results obtained in the field of combinatorial optimization can be found in [22]. Additionally, recent theoretical studies have investigated the learning ability of evolutionary algorithms [9,26].…”