On the Generalizability of Programs Synthesized by Grammar-Guided Genetic Programming

Sobania, Dominik

doi:10.1007/978-3-030-72812-0_9

Cited by 14 publications

(6 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…When CBGP does find a solution on training data, we observe an exceedingly high rate of generalization to unseen test data. This phenomenon is in contrast to the comparatively low generalization rates of all other GP systems included in our comparison [5,8,24]. Furthermore, the solution programs found by CBGP are small, even without the use of typical genome simplification techniques.…”

Section: Discussionmentioning

confidence: 57%

Functional Code Building Genetic Programming

Pantridge,

Helmuth,

Spector

2022

Preprint

View full text Add to dashboard Cite

General program synthesis has become an important application area for genetic programming (GP), and for artificial intelligence more generally. Code Building Genetic Programming (CBGP) is a recently introduced GP method for general program synthesis that leverages reflection and first class specifications to support the evolution of programs that may use arbitrary data types, polymorphism, and functions drawn from existing codebases. However, neither a formal description nor a thorough benchmarking of CBGP have yet been reported. In this work, we formalize the method of CBGP using algorithms from type theory. Specially, we show that a functional programming language and a Hindley-Milner type system can be used to evolve type-safe programs using the process abstractly described in the original CBGP paper. Furthermore, we perform a comprehensive analysis of the search performance of this functional variant of CBGP compared to other contemporary GP program synthesis methods. CCS CONCEPTS• Software and its engineering → Genetic programming.

show abstract

Section: Discussionmentioning

confidence: 57%

Functional Code Building Genetic Programming

Pantridge,

Helmuth,

Spector

2022

Preprint

View full text Add to dashboard Cite

show abstract

“…However, recent work shows that for some benchmark problems with a low output cardinality (especially for problems with a Boolean return value) the solutions that are found on the training cases often generalize poorly to unseen test cases. In particular, the generalization rate is low on these benchmark problems when lexicase selection is used [62]. This is not surprising, as lexicase selection makes use of the individual training cases during the selection process.…”

Section: B Grammar-guided Gpmentioning

confidence: 97%

“…A possibility to prevent invalid solutions through an unsuccessful genotype-phenotype mapping is to use a tree representation which is already given by the grammar's structure. Hence, tree representations have been often used in recent work using grammar-guided GP approaches (i.a., they are used in [11], [61], and [62]).…”

Section: B Grammar-guided Gpmentioning

confidence: 99%

“…Just like for the stack-based approaches (see Sect. IV-A), on average best success rates are achieved using variants of lexicase selection [11], [63], [62], [64]. However, recent work shows that for some benchmark problems with a low output cardinality (especially for problems with a Boolean return value) the solutions that are found on the training cases often generalize poorly to unseen test cases.…”

Section: B Grammar-guided Gpmentioning

confidence: 99%

See 1 more Smart Citation

Recent Developments in Program Synthesis with Evolutionary Algorithms

Sobania¹,

Schweim²,

Rothlauf³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

The automatic generation of computer programs is one of the main applications with practical relevance in the field of evolutionary computation. With program synthesis techniques not only software developers could be supported in their everyday work but even users without any programming knowledge could be empowered to automate repetitive tasks and implement their own new functionality. In recent years, many novel program synthesis approaches based on evolutionary algorithms have been proposed and evaluated on common benchmark problems. Therefore, we identify in this work the relevant evolutionary program synthesis approaches and provide an in-depth analysis of their performance. The most influential approaches we identify are stack-based, grammar-guided, as well as linear genetic programming. Further, we find that these approaches perform well on benchmark problems if there is a simple mapping from the given input to the correct output. On problems where this mapping is complex, e.g., if the problem consists of several subproblems or requires iteration/recursion for a correct solution, results tend to be worse. Consequently, for future work, we encourage researchers not only to use a program's output for assessing the quality of a solution but also the way towards a solution (e.g., correctly solved sub-problems).

show abstract

“…GGGP can either use the derivation tree [75] or a linear genome [58] as its program repre-sentation, which can be mapped to a resulting program via the context-free grammar. Recently, some work on GGGP has paid attention to grammar design [27], generalizability [69] and the quality of generated code [37,70].…”

Section: Related Workmentioning

confidence: 99%

Iterative Genetic Improvement: Scaling Stochastic Program Synthesis

Yuan¹,

Banzhaf²

2022

Preprint

View full text Add to dashboard Cite

Program synthesis aims to automatically find programs from an underlying programming language that satisfy a given specification. While this has the potential to revolutionize computing, how to search over the vast space of programs efficiently is an unsolved challenge in program synthesis. In cases where large programs are required for a solution, it is generally believed that stochastic search has advantages over other classes of search techniques. Unfortunately, existing stochastic program synthesizers do not meet this expectation very well, suffering from the scalability issue. Here we propose a new framework for stochastic program synthesis, called iterative genetic improvement to overcome this problem, a technique inspired by the practice of the software development process.The key idea of iterative genetic improvement is to apply genetic improvement to improve a current reference program, and then iteratively replace the reference program by the best program found. Compared to traditional stochastic synthesis approaches, iterative genetic improvement can build up the complexity of programs incrementally in a more robust way. We evaluate the approach on two program synthesis domains: list manipulation and string transformation. Our empirical results indicate that this method has considerable advantages over several representative stochastic program synthesizer techniques, both in terms of scalability and of solution quality.

show abstract

On the Generalizability of Programs Synthesized by Grammar-Guided Genetic Programming

Cited by 14 publications

References 22 publications

Functional Code Building Genetic Programming

Functional Code Building Genetic Programming

Recent Developments in Program Synthesis with Evolutionary Algorithms

Iterative Genetic Improvement: Scaling Stochastic Program Synthesis

Contact Info

Product

Resources

About