Greenify: A Game with the Purpose of Test Data Generation for Unit Testing

Abstract

One of the most important, but tedious and costly tasks of software testing process is test data generation. Several methods for automating this task have been presented, yet due to their practical drawbacks, test data generation is still widely performed by humans in industry.In our previous work, we employed the notion of Game With A Purpose (GWAP) and introduced Rings as a GWAP to reduce time and costs of human-based test data generation and increase its appeal to engage even nontechnical people. In this paper, we propose a new game, called Greenify, with the purpose of test data generation so that it solves the main issues of Rings. The environment of this game is built based on a program's control flow graph. To evaluate the proposed approach, we designed several game levels based on six different C++ programs and gave them to volunteering players. The results show that in comparison to both conventional human-based approach and Rings, Greenify generates test data with less rime for all feasible paths of the given benchmark programs. In addition, Greenify identifies the smaller set of likely infeasible paths.

Test Data Generation for Program Units Using a Game with a Purpose

“…Rings and Greenify were early works employing GWAP in human-based test data generation with promising results [9,10]. However, they had certain problems, including limitations on the number of program inputs, scalability, etc.…”

“…The players should change the values of the sliders and checkboxes until the color of all connected bubbles turns green. The results indicated that Greenify outperformed Rings, the conventional human-based approach, and the random test data generation method in terms of path coverage percentage and elapsed time for generating test data [10]. However, Greenify could not solve the following challenges of Rings:…”

“…In Greenify [10], a prior version of the game is going to be presented in this paper, the paths of a unit's CFG are represented through a string of connected light bubbles, and the input parameters of the unit are displayed via sliders and checkboxes. The players should change the values of the sliders and checkboxes until the color of all connected bubbles turns green.…”

“…This element allows the players to add a new lock/switch/keyboard to the game. The details of our solution to simulate arrays have been presented by Moosavi et al [10].…”

“…As a rst attempt, we restricted our research scope to the test data generation of program units. This resulted in two games, called Rings [9] and Greenify [10], which are reviewed in Section 2. We demonstrated that by using the notion of GWAP for test data generation, the main problems of humanbased test data generation could be alleviated in this way because:…”