Evolutionary Generation of Whole Test Suites

Abstract

Abstract-Recent advances in software testing allow automatic derivation of tests that reach almost any desired point in the source code. There is, however, a fundamental problem with the general idea of targeting one distinct test coverage goal at a time: Coverage goals are neither independent of each other, nor is test generation for any particular coverage goal guaranteed to succeed. We present EVOSUITE, a search-based approach that optimizes whole test suites towards satisfying a coverage criterion, rather than generating distinct test cases directed towards distinct coverage goals. Evaluated on five open source libraries and an industrial case study, we show that EVOSUITE achieves up to 18 times the coverage of a traditional approach targeting single branches, with up to 44% smaller test suites.

A Parallel Genetic Algorithm Based on Hadoop MapReduce for the Automatic Generation of JUnit Test Suites

“…It would be also interesting to verify how other approaches for test suite generation (e.g., EvoSuite [16]) could be take advantages of parallel/distributed computation as the one described in this paper.…”

“…We employ a representation similar to [16] for each test case that contains a sequence of statements of length l for object creation, state change, and method invocation. In particular, each statement can be one of these types: primitive, constructor, field, and method [16].…”

“…In particular, it is organized in four packages containing 259 classes, for a total of 4605 Lines of Code (LOC) and 1446 branches (see Table I). As done in previous work [16], we tested all the classes which were not interface, abstract, or private (see Table II). …”

“…Search-based algorithms have been used to automate a variety of software testing activities, such as test data generation (e.g., [2][21] [24][36]), test case generation and selection (e.g., [16][39]), test case prioritization (e.g., [38]), and so on. For sake of space we remind interested readers to [1] and [27] for a review of these works and we discuss in the following the parallel Search-based approaches proposed for Software Testing.…”

Semi-automatic Search-Based Test Generation

“…ArgsParser argsParser0 = new ArgsParser(); String string0 = "</xml>"; ⇐ "--xml" String[] stringArray0 = new String [4]; stringArray0 [1] = string0; argsParser0.setArgs(stringArray0); argsParser.parseSwitchArgument(string0); ⇐ "xml" Figure 2. Example test case generated by EVOSUITE (the string </xml> results from seeding), almost reaching the target branch, yet the last if statement evaluates to false.…”