Improving Test Generation under Rich Contracts by Tight Bounds and Incremental SAT Solving

Abad, Pablo Garrido; Aguirre, Nazareno; Bengolea, Valeria; Ciolek, Daniel; Frias, Marcelo F.; Galeotti, Juan Pablo; Maibaum, Tom; Moscato, Mariano M.; Rosner, Nicolás; Vissani, Ignacio

doi:10.1109/icst.2013.46

Cited by 34 publications

(33 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Many test generation tools only handle basic coverage criteria, such as bounded path coverage or decision coverage. Three interesting exceptions to be compared with our work are Fshell [30], FAJITA [31] and Apex [32].…”

Section: Related Workmentioning

confidence: 76%

Taming Coverage Criteria Heterogeneity with LTest

Marcozzi¹,

Bardin²,

Delahaye³

et al. 2017

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)

View full text Add to dashboard Cite

Abstract-Automated white-box testing is a major issue in software engineering. In previous work, we introduced LTest, a generic and integrated toolkit for automated white-box testing of C programs. LTest supports a broad class of coverage criteria in a unified way (through the label specification mechanism) and covers most major parts of the testing process -including coverage measurement, test generation and detection of infeasible test objectives. However, the original version of LTest was unable to handle several major classes of coverage criteria, such as MCDC or dataflow criteria. Moreover, its practical applicability remained barely assessed.In this work, we present a significantly extended version of LTest that supports almost all existing testing criteria, including MCDC and some software security properties, through a native support of recently proposed hyperlabels. We also provide a more realistic view on the practical applicability of the extended tool, with experiments assessing its efficiency and scalability on realworld programs.

show abstract

Section: Related Workmentioning

confidence: 76%

Taming Coverage Criteria Heterogeneity with LTest

Marcozzi¹,

Bardin²,

Delahaye³

et al. 2017

2017 IEEE International Conference on Software Testing, Verification and Validation (ICST)

View full text Add to dashboard Cite

show abstract

“…Test input generation tools are no exception -in order to automatically generate tests, these tools often require a specification of the valid inputs of the program under consideration [1,2,5,14,21,32]. Several different approaches exist for expressing input specifications, which in many cases can become quite intricate and complex to express.…”

Section: Hybrid Input Specificationsmentioning

confidence: 99%

“…Others, in particular test input generation, are typically very hard to automate. Despite the inherent complexity of automating test input generation, various techniques and tools have been proposed to automatically produce test inputs, including some based on random generation [5,26] as well as others based on several different forms of constraint solving or model checking [1,2,14,21,32].…”

Section: Introductionmentioning

confidence: 99%

Bounded exhaustive test input generation from hybrid invariants

Rosner

Bengolea²,

Aguirre³

et al. 2014

Proceedings of the 2014 ACM International Conference on Object Oriented Programming Systems Languages &Amp; Applications

Self Cite

View full text Add to dashboard Cite

We present a novel technique for producing bounded exhaustive test suites from hybrid invariants, i.e., invariants that are expressed imperatively, declaratively, or as a combination of declarative and imperative predicates. Hybrid specifications are processed using known mechanisms for the imperative and declarative parts, but combined in a way that enables us to exploit information from the declarative side, such as tight bounds computed from the declarative specification, to improve the search both on the imperative and declarative sides. Moreover, our technique automatically evaluates different possible ways of processing the imperative side, and the alternative settings (imperative or declarative) for parts of the invariant available both declaratively and imperatively, to decide the most convenient invariant configuration with respect to efficiency in test generation. This is achieved by transcoping, i.e., by assessing the efficiency of the different alternatives on small scopes (where generation times are negligible), and then extrapolating the results to larger scopes.We also show experiments involving collection classes that support the effectiveness of our technique, by demonstrating that (i) bounded exhaustive suites can be computed from hybrid invariants significantly more efficiently than doing so using state-of-the-art purely imperative and purely declarative approaches, and (ii) our technique is able to automatically determine efficient hybrid invariants, in the sense that they lead to an efficient computation of bounded exhaustive suites, using transcoping.

show abstract

“…Abad et al [24] developed a new test generator, called FAJITA. They compared the branch coverage and performance of FAJITA with Pex, Kiasan, Randoop, AutoTest, and EvoSuite.…”

Section: ) Improving Test Generation Under Rich Contracts By Tight Bmentioning

confidence: 99%

Validation of Software Testing Experiments: A Meta-Analysis of ICST 2013

Hays

Hayes

Bathke

2014

2014 IEEE Seventh International Conference on Software Testing, Verification and Validation

View full text Add to dashboard Cite

Abstract-Researchers in software testing are often faced with the following problem of empirical validation: does a new testing technique actually help analysts find more faults than some baseline method? Researchers evaluate their contribution using statistics to refute the null hypothesis that their technique is no better at finding faults than the state of the art. The decision as to which statistical methods are appropriate is best left to an expert statistician, but the reality is that software testing researchers often don't have this luxury. We developed an algorithm, MeansTest, to help automate some aspects of statistical analysis. We implemented MeansTest in the statistical software environment R, encouraging reuse and decreasing the need to write and test statistical analysis code. Our experiment showed that MeansTest has significantly higher F-measures than several other common hypothesis tests. We applied MeansTest to systematically validate the work presented at the 2013 IEEE Sixth International Conference on Software Testing, Verification, and Validation (ICST'13). We found six papers that potentially misstated the significance of their results. MeansTest provides a free and easy-to-use possibility for researchers to check whether their chosen statistical methods and the results obtained are plausible. It is available for download at coest.org.

show abstract

Improving Test Generation under Rich Contracts by Tight Bounds and Incremental SAT Solving

Cited by 34 publications

References 19 publications

Taming Coverage Criteria Heterogeneity with LTest

Taming Coverage Criteria Heterogeneity with LTest

Bounded exhaustive test input generation from hybrid invariants

Validation of Software Testing Experiments: A Meta-Analysis of ICST 2013

Contact Info

Product

Resources

About