Handling test length bloat

Arcuri, Andrea

doi:10.1002/stvr.1495

Cited by 18 publications

(6 citation statements)

References 40 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is so that the initial execution of the predicate, with some specific true/false evaluation, is not lost in the process of pursuing the alternative outcome. As longer test suites require more memory and execution time, controlling the length of the test suite can improve search performance [9]. Therefore, when deciding which test suites should proceed into the population for the next iteration of the search, EvoSuite prefers shorter test suites to test suites with the same fitness but whose test cases of composed of cumulatively higher number of statements.…”

Section: Genetic Algorithm Search For Test Suitesmentioning

confidence: 98%

Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?

Shamshiri

Rojas

Fraser

et al. 2015

Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

Achieving high structural coverage is an important aim in software testing. Several search-based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well-established arguments behind using evolutionary search algorithms (e.g., genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object-oriented classes. In this paper, we report an empirical study on the effects of using a genetic algorithm (GA) to generate test suites over generating test suites incrementally with random search, by applying the EvoSuite unit test suite generator to 1,000 classes randomly selected from the SF110 corpus of open source projects. Surprisingly, the results show little difference between the coverage achieved by test suites generated with evolutionary search compared to those generated using random search. A detailed analysis reveals that the genetic algorithm covers more branches of the type where standard fitness functions provide guidance. In practice, however, we observed that the vast majority of branches in the analyzed projects provide no such guidance.

show abstract

Section: Genetic Algorithm Search For Test Suitesmentioning

confidence: 98%

Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?

Shamshiri

Rojas

Fraser

et al. 2015

Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…However, the implementation of the GA in the underlying EVOSUITE ranks two individuals by their size if their fitness values are equal, such that smaller test sets have a higher probability of being selected for reproduction. Consequently, during phases of the search where there is no exploration of new coverage or adequacy, redundant test cases are removed from the test sets, and thus behavioural adequacy does not lead to bloat .…”

Section: Discussionmentioning

confidence: 88%

“…Out of this population, the GA selects individuals for reproduction, where individuals with better fitness values have a higher probability of being selected. To avoid undesired growth of the population (bloat ), individuals with identical fitness are ranked by size, such that smaller test sets are more likely to be selected. Crossover and mutation are applied with given probabilities, where crossover exchanges individual tests between two parent test sets, and mutation changes existing tests or adds new tests to a test set.…”

Section: Generating Adequate Test Setsmentioning

confidence: 99%

Assessing and generating test sets in terms of behavioural adequacy

Walkinshaw

2015

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

SUMMARYIdentifying a finite test set that adequately captures the essential behaviour of a program such that all faults are identified is a well-established problem. This is traditionally addressed with syntactic adequacy metrics (e.g. branch coverage), but these can be impractical and may be misleading even if they are satisfied. One intuitive notion of adequacy, which has been discussed in theoretical terms over the past three decades, is the idea of behavioural coverage: If it is possible to infer an accurate model of a system from its test executions, then the test set can be deemed to be adequate. Despite its intuitive basis, it has remained almost entirely in the theoretical domain because inferred models have been expected to be exact (generally an infeasible task) and have not allowed for any pragmatic interim measures of adequacy to guide test set generation. This paper presents a practical approach to incorporate behavioural coverage. Our BESTEST approach (1) enables the use of machine learning algorithms to augment standard syntactic testing approaches and (2) shows how search-based testing techniques can be applied to generate test sets with respect to this criterion. An empirical study on a selection of Java units demonstrates that test sets with higher behavioural coverage significantly outperform current baseline test criteria in terms of detected faults.

show abstract

“…As longer test suites require more memory and execution time, controlling the length of the test suite can improve search performance . Therefore, when deciding which test suites should proceed into the population for the next iteration of the search, E VO S UITE prefers shorter test suites to test suites with the same fitness but are composed of a higher number of statements.…”

Section: Search‐based Test Generationmentioning

confidence: 99%

“…As longer test suites require more memory and execution time, controlling the length of the test suite can improve search performance [35].…”

Section: Genetic Algorithm Search For Test Suitesmentioning

confidence: 99%

Random or evolutionary search for object‐oriented test suite generation?

Shamshiri

Rojas

Gazzola

et al. 2018

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Summary An important aim in software testing is constructing a test suite with high structural code coverage, that is, ensuring that most if not all of the code under test have been executed by the test cases comprising the test suite. Several search‐based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well‐established arguments behind using evolutionary search algorithms (eg, genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object‐oriented classes. In this paper, we report an empirical study on the effects of using evolutionary algorithms (including a genetic algorithm and chemical reaction optimization) to generate test suites, compared with generating test suites incrementally with random search. We apply the EVOSUITE unit test suite generator to 1000 classes randomly selected from the SF110 corpus of open‐source projects. Surprisingly, the results show that the difference is much smaller than one might expect: While evolutionary search covers more branches of the type where standard fitness functions provide guidance, we observed that, in practice, the vast majority of branches do not provide any guidance to the search. These results suggest that, although evolutionary algorithms are more effective at covering complex branches, a random search may suffice to achieve high coverage of most object‐oriented classes.

show abstract

Handling test length bloat

Cited by 18 publications

References 40 publications

Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?

Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?

Assessing and generating test sets in terms of behavioural adequacy

Random or evolutionary search for object‐oriented test suite generation?

Contact Info

Product

Resources

About