A Divergence-Oriented Approach to Adaptive Random Testing of Java Programs

Abstract

Abstract-Adaptive Random Testing (ART) is a testing technique which is based on an observation that a test input usually has the same potential as its neighbors in detection of a specific program defect. ART helps to improve the efficiency of random testing in that test inputs are selected evenly across the input spaces. However, the application of ART to objectoriented programs (e.g., C++ and Java) still faces a strong challenge in that the input spaces of object-oriented programs are usually high dimensional, and therefore an even distribution of test inputs in a space as such is difficult to achieve. In this paper, we propose a divergence-oriented approach to adaptive random testing of Java programs to address this challenge. The essential idea of this approach is to prepare for the tested program a pool of test inputs each of which is of significant difference from the others, and then to use the ART technique to select test inputs from the pool for the tested program. We also develop a tool called ARTGen to support this testing approach, and conduct experiment to test several popular opensource Java packages to assess the effectiveness of the approach. The experimental result shows that our approach can generate test cases with high quality.

A Similarity Metric for the Inputs of OO Programs and Its Application in Adaptive Random Testing

“…We have carefully designed the experiments such that OMISS-ART can be fairly compared with other existing testing approaches. After reimplementing ARTOO and divergence-oriented ART using .NET, we validated their correctness against the examples and results in the original papers [17], [18]. However, the conclusions from our empirical studies are based on 5100 datasets (300 runs for 17 subject programs), and since the datasets are larger than those in the original empirical studies [17], [18], the margins of error for the 95% confidence interval reported in Tables VI and XI are tighter. The second potential threat relates to the selection of subject programs.…”

“…ART has drawn a lot of attention, both from academia and from industry, and a number of different algorithms have been developed [13], [18], [19], [24]- [30], with one of the most popular being FSCS-ART [19]. With FSCS-ART, previously executed test inputs are stored in a set T, and whenever a new test input is needed, a fixed number of random inputs are generated as a candidate set, C, from which, based on some selection criteria, the best candidate is then chosen.…”

“…ART [18] were used in this study: create-deep was set to 4; and both max-call and max-diversify were set to 5. The createdeep setting is the depth of recursively created reference objects; max-call is the maximum number of calls to the same public method of the same object; and max-diversify is the maximum number of calls to different public methods in the same object.…”

“…However, this metric can only choose one object at a time as a test input to test one specific method of the class. Lin et al [18] tried to improve ARTOO by invoking extra methods of the chosen object before running the specific method. Using the Java language, they implemented this approach (called divergence-oriented ART) in a new tool called ARTGen, which was found to outperform RT in terms of finding faults.…”

Ballerina: Automatic generation and clustering of efficient random unit tests for multithreaded code

“…Our BALLERINA technique utilizes Randoop [14] and modifies it to more densely cover states of objects for the given CUT. Related to this, techniques based on adaptive random testing [43]- [45] use various measures for object distance to generate more divergent test inputs. However, unlike those projects, BALLERINA generates tests for multithreaded code.…”