Test Case Generation by Symbolic Execution: Basic Concepts, a CLP-Based Instance, and Actor-Based Concurrency

Albert, Elvira; Arenas, Puri; Gómez-Zamalloa, Miguel; Rojas, José Miguel

doi:10.1007/978-3-319-07317-0_7

Cited by 11 publications

(6 citation statements)

References 48 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…In our study, we rely on two different test data generation tools: EVOSUITE , based on genetic algorithms, and RANDOOP , which implements a random testing approach. Despite we rely on the most widely used tools in practice, we cannot ensure the applicability or our findings to different generation approaches such as AVM or symbolic execution …”

Section: Threats To Validitymentioning

confidence: 88%

Branch coverage prediction in automated testing

Grano

Titov

Panichella

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

Software testing is crucial in continuous integration (CI). Ideally, at every commit, all the test cases should be executed, and moreover, new test cases should be generated for the new source code. This is especially true in a Continuous Test Generation (CTG) environment, where the automatic generation of test cases is integrated into the continuous integration pipeline. In this context, developers want to achieve a certain minimum level of coverage for every software build. However, executing all the test cases and, moreover, generating new ones for all the classes at every commit is not feasible. As a consequence, developers have to select which subset of classes has to be tested and/or targeted by test-case generation. We argue that knowing a priori the branch coverage that can be achieved with test-data generation tools can help developers into taking informed decision about those issues. In this paper, we investigate the possibility to use source-code metrics to predict the coverage achieved by test-data generation tools. We use four different categories of source-code features and assess the prediction on a large data set involving more than 3'000 Java classes. We compare different machine learning algorithms and conduct a fine-grained feature analysis aimed at investigating the factors that most impact the prediction accuracy. Moreover, we extend our investigation to four different search budgets. Our evaluation shows that the best model achieves an average 0.15 and 0.21 MAE on nested cross-validation over the different budgets, respectively, on EVOSUITE and RANDOOP. Finally, the discussion of the results demonstrate the relevance of coupling-related features for the prediction accuracy. KEYWORDS automated software testing, coverage prediction, machine learning, software testing INTRODUCTIONSoftware testing is widely recognized as a crucial task in any software development process, 1 estimated at being at least about half of the entire development cost. 2,3 In the last years, we witnessed a wide adoption of continuous integration (CI) practices, where new or changed code is integrated extremely frequently into the main codebase. Testing plays an important role in such a pipeline: In an ideal world, at every single commit, every system's test case should be executed (regression testing). Moreover, additional test cases might be automatically generated to test all the new -or modified-code introduced into the main codebase. 4 This is especially true in a Continuous Test Generation (CTG) environment, where the generation of test cases is directly integrated into the continuous integration cycle. 4 However, because of the time constraints between frequent commits, a complete regression testing is not feasible for large projects. 5 Furthermore, even test suite augmentation, 6 ie, the automatic generation considering code changes and their effect on the previous codebase, is hardly doable because of the extensive amount of time needed to generate tests for just a single class. As developers want to ensure a cer...

show abstract

Section: Threats To Validitymentioning

confidence: 88%

Branch coverage prediction in automated testing

Grano

Titov

Panichella

et al. 2019

J Software Evolu Process

View full text Add to dashboard Cite

show abstract

“…In our study we rely on two different test data generation tools: EvoSuite, based on genetic algorithms, and Randoop, which implements a random testing approach. Despite we rely on the most widely used tools in practice, we cannot ensure the applicability or our findings to different generation approaches such as AVM [25] or symbolic execution [1].…”

Section: Threats To Validitymentioning

confidence: 98%

How high will it be? Using machine learning models to predict branch coverage in automated testing

Grano

Titov

Panichella

et al. 2018

2018 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE)

View full text Add to dashboard Cite

Software testing is a crucial component in modern continuous integration development environment. Ideally, at every commit, all the system's test cases should be executed and moreover, new test cases should be generated for the new code. This is especially true in the a Continuous Test Generation (CTG) environment, where the automatic generation of test cases is integrated into the continuous integration pipeline. Furthermore, developers want to achieve a minimum level of coverage for every build of their systems. Since both executing all the test cases and generating new ones for all the classes at every commit is not feasible, they have to select which subset of classes has to be tested. In this context, knowing a priori the branch coverage that can be achieved with test data generation tools might gives some useful indications for answering such a question. In this paper, we take the first steps towards the definition of machine learning models to predict the branch coverage achieved by test data generation tools. We conduct a preliminary study considering well known code metrics as a features. Despite the simplicity of these features, our results show that using machine learning to predict branch coverage in automated testing is a viable and feasible option.

show abstract

“…In most cases, however, an infinite set of symbolic states will be produced, and symbolic execution will be non-terminating even on terminating programs. A simple solution is to restrict the number of loop iterations by a limit N given as a parameter of the symbolic execution, resulting in a N -bounded symbolic execution, also called loop-k [1]. When using a loop limit, we would like to distinguish between the executions that terminate normally and executions that reach the loop limit.…”

Section: Introductionmentioning

confidence: 99%

Ghost Code in Action: Automated Verification of a Symbolic Interpreter

Becker

Marché

2020

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Symbolic execution is a basic concept for the static analysis of programs. It amounts to representing sets of concrete program states as a logical formula relating the program variables, and interpreting sets of executions as a transformation of that formula. We are interested in formalising the correctness of a symbolic interpreter engine, expressed by an over-approximation property stating that symbolic execution covers all concrete executions, and an under-approximation property stating that no useless symbolic states are generated. Our formalisation is tailored for automated verification, that is the automated discharge of verification conditions to SMT solvers. To achieve this level of automation, we appropriately annotate the code of the symbolic interpreter with an original use of both ghost data and ghost statements.

show abstract

Test Case Generation by Symbolic Execution: Basic Concepts, a CLP-Based Instance, and Actor-Based Concurrency

Cited by 11 publications

References 48 publications

Branch coverage prediction in automated testing

Branch coverage prediction in automated testing

How high will it be? Using machine learning models to predict branch coverage in automated testing

Ghost Code in Action: Automated Verification of a Symbolic Interpreter

Contact Info

Product

Resources

About