Reducing the Cost of Model-Based Testing through Test Case Diversity

Hemmati, Hadi; Arcuri, Andrea; Briand, Lionel C.

doi:10.1007/978-3-642-16573-3_6

Cited by 32 publications

(28 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A similar objective is pursued by H. Hemmati and his colleagues (Hemmati et al 2010) who propose a test suite reduction algorithm based on the similarity of test cases. Their study concludes that "Test cases that find the same faults tend to be more similar to each other than with other test cases", and that "test cases that find different faults tend to be more different from each other than test cases that find the same faults".…”

Section: Related Workmentioning

confidence: 95%

Prioritizing test cases with string distances

et al. 2011

View full text Add to dashboard Cite

Test case prioritisation aims at finding an ordering which enhances a certain property of an ordered test suite. Traditional techniques rely on the availability of code or a specification of the program under test. We propose to use string distances on the text of test cases for their comparison and elaborate a prioritisation algorithm. Such a prioritisation does not require code or a specification and can be useful for initial testing and in cases when code is difficult to instrument. In this paper, we also report on experiments performed on the "Siemens Test Suite", where the proposed prioritisation technique was compared with random permutations and four classical string distance metrics were evaluated. The obtained results, confirmed by a statistical analysis, indicate that prioritisation based on string distances is more efficient in finding defects than random ordering of the test suite: the test suites prioritized using string distances are more efficient in detecting the strongest mutants, and, on average, have a better APFD than randomly ordered test suites. The results suggest that string distances can be used for prioritisation purposes, and Manhattan distance could be the best choice.

show abstract

Section: Related Workmentioning

confidence: 95%

Prioritizing test cases with string distances

et al. 2011

View full text Add to dashboard Cite

show abstract

“…This practically entails that a tester cannot, in many circumstances, run as many test cases as ideally needed. For example, this is a case we encountered when applying model-based testing on the video-conference system, for which we had to develop sophisticated techniques to choose subsets of test cases that could be run within the testing budget [20], [21]. Considering the case in Table 3, a testing budget of 400 test cases would be significantly more than what CIT yields with t ¼ 3 (181) but much less than what is obtained with t ¼ 4 (924).…”

Section: When Information Is Required About Fault Detectionmentioning

confidence: 99%

Formal Analysis of the Probability of Interaction Fault Detection Using Random Testing

Arcuri

Briand

2012

IIEEE Trans. Software Eng.

Self Cite

View full text Add to dashboard Cite

Abstract-Modern systems are becoming highly configurable to satisfy the varying needs of customers and users. Software product lines are hence becoming a common trend in software development to reduce cost by enabling systematic, large-scale reuse. However, high levels of configurability entail new challenges. Some faults might be revealed only if a particular combination of features is selected in the delivered products. But testing all combinations is usually not feasible in practice, due to their extremely large numbers. Combinatorial testing is a technique to generate smaller test suites for which all combinations of t features are guaranteed to be tested. In this paper, we present several theorems describing the probability of random testing to detect interaction faults and compare the results to combinatorial testing when there are no constraints among the features that can be part of a product. For example, random testing becomes even more effective as the number of features increases and converges toward equal effectiveness with combinatorial testing. Given that combinatorial testing entails significant computational overhead in the presence of hundreds or thousands of features, the results suggest that there are realistic scenarios in which random testing may outperform combinatorial testing in large systems. Furthermore, in common situations where test budgets are constrained and unlike combinatorial testing, random testing can still provide minimum guarantees on the probability of fault detection at any interaction level. However, when constraints are present among features, then random testing can fare arbitrarily worse than combinatorial testing. As a result, in order to have a practical impact, future research should focus on better understanding the decision process to choose between random testing and combinatorial testing, and improve combinatorial testing in the presence of feature constraints.

show abstract

“…ART has been used in testing Object-Oriented software, aspectoriented software and embedded software [24]; for test case prioritization [34], model based test case selection [35], and enlarged and high dimensional input domain [36]. Limitations of ART have been examined [37], including studying a lower bound on the Fmeasure achievable by any test method; ART's F-measure is close to the lower bound [38].…”

Section: Related Research On Artmentioning

confidence: 99%