Adaptive random testing based on distribution metrics

Chen, Tsong Yueh; Kuo, Fei-Ching; Liu, Huai

doi:10.1016/j.jss.2009.05.017

Cited by 37 publications

(14 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Two complex numerical applications (Gammq and Expint) are also employed in this empirical study, which have been previously used in the literature of software testing (e.g. 32). Table II summarizes their properties.…”

Section: Analysis Of Normalizing Functionsmentioning

confidence: 99%

It really does matter how you normalize the branch distance in search‐based software testing

Arcuri

2011

Software Testing Verif & Rel

View full text Add to dashboard Cite

SUMMARY The use of search algorithms for test data generation has seen many successful results. For structural criteria like branch coverage, heuristics have been designed to help the search. The most common heuristic is the use of approach level (usually represented with an integer) to reward test cases whose executions get close (in the control flow graph) to the target branch. To solve the constraints of the predicates in the control flow graph, the branch distance is commonly employed. These two measures are linearly combined. Since the approach level is more important, the branch distance is normalized, often in the range [0, 1]. In this paper, different types of normalizing functions are analyzed. The analyses show that the one that is usually employed in the literature has several flaws. The paper presents a different normalizing function that is very simple and does not suffer from these limitations. Empirical and analytical analyses are carried out to compare these two functions. In particular, their effect is studied on commonly used search algorithms, such as Hill Climbing, Simulated Annealing and Genetic Algorithms. Copyright © 2011 John Wiley & Sons, Ltd.

show abstract

Section: Analysis Of Normalizing Functionsmentioning

confidence: 99%

It really does matter how you normalize the branch distance in search‐based software testing

Arcuri

2011

Software Testing Verif & Rel

View full text Add to dashboard Cite

show abstract

“…ART has drawn a lot of attention, both from academia and from industry, and a number of different algorithms have been developed [13], [18], [19], [24]- [30], with one of the most popular being FSCS-ART [19]. With FSCS-ART, previously executed test inputs are stored in a set T, and whenever a new test input is needed, a fixed number of random inputs are generated as a candidate set, C, from which, based on some selection criteria, the best candidate is then chosen.…”

Section: B Adaptive Random Testingmentioning

confidence: 99%

“…In order to improve the failure-detection effectiveness of RT, Chen et al proposed an enhancement, adaptive random testing (ART) [4], [13], [14], which was inspired by empirical observations that many faulty programs have contiguous regions of failure-causing inputs [15]. To quickly find a failure in this situation, ART selects randomly generated follow-up test inputs further away from the previously executed, nonfailure-causing test inputs.…”

mentioning

confidence: 99%

“…To quickly find a failure in this situation, ART selects randomly generated follow-up test inputs further away from the previously executed, nonfailure-causing test inputs. ART has been found to consistently outperform RT for fault detection [13], [15]. One challenge faced when applying ART is how to measure the difference between inputs (named "distance" hereafter).…”

mentioning

confidence: 99%

See 1 more Smart Citation

A Similarity Metric for the Inputs of OO Programs and Its Application in Adaptive Random Testing

et al. 2017

Self Cite

View full text Add to dashboard Cite

Abstract-Random testing (RT) has been identified as one of the most popular testing techniques, due to its simplicity and ease of automation. Adaptive random testing (ART) has been proposed as an enhancement to RT, improving its fault-detection effectiveness by evenly spreading random test inputs across the input domain. To achieve the even spreading, ART makes use of distance measurements between consecutive inputs. However, due to the nature of object-oriented software (OOS), its distance measurement can be particularly challenging: Each input may involve multiple classes, and interaction of objects through method invocations. Two previous studies have reported on how to test OOS at a single-class level using ART. In this study, we propose a new similarity metric to enable multiclass level testing using ART. When generating test inputs (for multiple classes, a series of objects, and a sequence of method invocations), we use the similarity metric to calculate the distance between two series of objects, and between two sequences of method invocations. We integrate this metric with ART and apply it to a set of open-source OO programs, with the empirical results showing that our approach outperforms other RT and ART approaches in OOS testing.

show abstract

“…The simulation exercises were repeated 1,000 times and the experimental results of the computation times of the algorithms are reported in Table II. In addition to computational complexity, diversity measurement metrics, namely, discrepancy and dispersion [5], were also used to measure the effectiveness of the divide-andconquer technique. The dispersion metric indicates whether there is a large empty region in input domain D, which is reflected by the maximum distance of any test case from its nearest neighbor.…”

Section: (B) and (D)mentioning

confidence: 99%

The ART of Divide and Conquer: An Innovative Approach to Improving the Efficiency of Adaptive Random Testing

Chow

Chen

Tse

2013

2013 13th International Conference on Quality Software

Self Cite

View full text Add to dashboard Cite

Abstract-Test case selection is a prime process in the engineering of test harnesses. In particular, test case diversity is an important concept. In order to achieve an even spread of test cases across the input domain, Adaptive Random Testing (ART) was proposed such that the history of previously executed test cases are taken into consideration when selecting the next test case. This was achieved through various means such as best candidate selection, exclusion, , and diversity metrics. Empirical studies showed that ART algorithms make good use of the concept of even spreading and achieve 40 to 50% improvement in test effectiveness over random testing in revealing the first failure, which is close to the theoretical limit. However, the computational complexity of ART algorithms may be quadratic or higher, and hence efficiency is an issue when a large number of previously executed test cases are involved. This paper proposes an innovative divide-and-conquer approach to improve the efficiency of ART algorithms while maintaining their performance in effectiveness. Simulation studies have been conducted to gauge its efficiency against two most commonly used ART algorithms, namely, fixed size candidate set and restricted random testing. Initial experimental results show that the divideand-conquer technique can provide much better efficiency while maintaining similar, or even better, effectiveness. *

show abstract

Adaptive random testing based on distribution metrics

Cited by 37 publications

References 26 publications

It really does matter how you normalize the branch distance in search‐based software testing

It really does matter how you normalize the branch distance in search‐based software testing

A Similarity Metric for the Inputs of OO Programs and Its Application in Adaptive Random Testing

The ART of Divide and Conquer: An Innovative Approach to Improving the Efficiency of Adaptive Random Testing

Contact Info

Product

Resources

About