Factors affecting the use of genetic algorithms in test suite augmentation

Abstract

Test suite augmentation techniques are used in regression testing to help engineers identify code elements affected by changes, and generate test cases to cover those elements. Researchers have created various approaches to identify affected code elements, but only recently have they considered integrating, with this task, approaches for generating test cases. In this paper we explore the use of genetic algorithms in test suite augmentation. We identify several factors that impact the effectiveness of this approach, and we present the results of a case study exploring the effects of one of these factors: the manner in which existing and newly generated test cases are utilized by the genetic algorithm. Our results reveal several ways in which this factor can influence augmentation results, and reveal open problems that researchers must address if they wish to create augmentation techniques that make use of genetic algorithms.

KATCH: high-coverage testing of software patches

“…katch also shares characteristics with search-based software testing (SBST) [16,40,43]. First, our notion of estimated distance is similar to that of fitness in SBST.…”

“…Synthesising inputs which cover a target is an essential problem in test generation and debugging and has been addressed through a variety of techniques, including symbolic execution, dependence analysis, iterative relaxation and search-based software testing, among others [2,15,21,36,40,41,44].…”

Search-based inference of polynomial metamorphic relations

“…Search-based optimization techniques have been widely applied to software testing, including test-suite generation [8,4,56,24,53] and optimization [72,31,43,3,74]. Besides software testing, search-based optimization techniques have also been applied to fault localization [65], program analysis [76], software refactoring [29,30,55], cost estimation [19], project scheduling [1,18], decisions design optimization [10], automated negotiation [17], source code parallelization [57], requirement engineering [27,64], variability management [41], and so on.…”

High-Coverage Symbolic Patch Testing

“…Synthesizing inputs which cover a target is an essential problem in automated test generation and debugging [1,16,27,29,30,32]. While we borrow ideas from the state of the art in these areas, and combine symbolic execution, static analysis and various heuristics, our approach differs by treating the task as an optimization problem with the goal of exploring paths that minimize the estimated distance to the target.…”

“…A different approach for covering specific program points is to use genetic algorithms [12,29]. Such algorithms usually encode program paths as binary strings, each bit representing the outcome of a branch condition evaluation, and then define a fitness function and crossover and mutation operators operating on this encoding.…”