Test Data Generation for Program Units Using a Game with a Purpose

Abstract

Today, several methods have been presented to automate test data generation; because of the low maturity level of automatic methods, this is still widely carried out by humans in industry. Hence, the challenge is nding approaches in which humans could generate test data through more attractive, faster, and cheaper ways. To achieve this, one approach is using a Game With A Purpose (GWAP) in test data generation. In our previous work, we introduced two games called Rings and Greenify, by which many inexpensive players with no special technical abilities become engaged in test data generation. This paper presents an entirely new GWAP for test data generation called Quest Of Treasure Explorer (QOTE). QOTE provides a di erent gameplay and has certain advantages compared to prior games, including faster generation of test data, easier puzzles, narration, etc. Experimental results have shown that QOTE outperforms prior games in two aspects: game quality and capability of test data generation. We have also conducted an experiment based on mutation analysis to further evaluate the test data generation capabilities of QOTE compared to four automatic approaches. According to this experiment, QOTE outperforms the four competitors regarding average mutation scores.