Combining symbolic execution and search-based testing for programs with complex heap inputs

Braione, Pietro; Denaro, Giovanni; Mattavelli, Andrea; Pezzè, Mauro

doi:10.1145/3092703.3092715

Cited by 55 publications

(48 citation statements)

References 53 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…Generating complex input objects is a challenge faced by many automated test generation approaches, including search-based software testing and symbolic execution (Braione et al 2017). Usually, the input space of each input is large and generating proper data enabling the search process to cover its goals is difficult.…”

Section: Input Data Generationmentioning

confidence: 99%

A benchmark-based evaluation of search-based crash reproduction

et al. 2019

View full text Add to dashboard Cite

Crash reproduction approaches help developers during debugging by generating a test case that reproduces a given crash. Several solutions have been proposed to automate this task. However, the proposed solutions have been evaluated on a limited number of projects, making comparison difficult. In this paper, we enhance this line of research by proposing JCrashPack, an extensible benchmark for Java crash reproduction, together with ExRunner, a tool to simply and systematically run evaluations. JCrashPack contains 200 stack traces from various Java projects, including industrial open source ones, on which we run an extensive evaluation of EvoCrash, the state-of-the-art tool for search-based crash reproduction. EvoCrash successfully reproduced 43% of the crashes. Furthermore, we observed that reproducing NullPointerException, IllegalArgumentException, and IllegalStateException is relatively easier than reproducing ClassCastException, ArrayIndexOutOfBoundsException and StringIndexOutOfBoundsException. Our results include a detailed manual analysis of EvoCrash outputs, from which we derive 14 current challenges for crash reproduction, among which the generation of input data and the handling of abstract and anonymous classes are the most frequents. Finally, based on those challenges, we discuss future research directions for search-based crash reproduction for Java.

show abstract

Section: Input Data Generationmentioning

confidence: 99%

A benchmark-based evaluation of search-based crash reproduction

et al. 2019

View full text Add to dashboard Cite

show abstract

“…In case the path condition implies v is constrained by a points-to predicate, it substitutes v.f i with the corresponding symbolic name for the field in the predicate (lines 8-9). Otherwise, if v is constrained by an inductive predicate, it unfolds the predicate to find points-to predicate for v (lines [10][11][12][13][14]. In the last case (lines [15][16], it considers the current path condition does not have enough information to resolve v.f i and simply returns empty.…”

Section: Concolic Executionmentioning

confidence: 99%

“…That is, the users provide a program and a precondition, then apply CSF to automatically generate a set of test inputs. The experimental subject is a comprehensive set of benchmark programs collected from previous publications, which includes Singly-Linked List (SLL), Doubly-Linked List (DLL), Stack, Binary Search Tree (BST), Red Black Tree (RBT) from SIR [7], AVL Tree, AA Tree (AAT) from Sireum/Kiasan [8], Tll from [27], the motivation example from SUSHI [10], the TSAFE project [17], and the Gantt project [3]. In total, we have 74 methods whose line of codes range from dozens to more than one thousand.…”

Section: First Experimentsmentioning

confidence: 99%

Concolic Testing Heap-Manipulating Programs

Pham

Phan

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Concolic testing is a test generation technique which works effectively by integrating random testing generation and symbolic execution. Existing concolic testing engines focus on numeric programs. Heap-manipulating programs make extensive use of complex heap objects like trees and lists. Testing such programs is challenging due to multiple reasons. Firstly, test inputs for such programs are required to satisfy non-trivial constraints which must be specified precisely. Secondly, precisely encoding and solving path conditions in such programs are challenging and often expensive. In this work, we propose the first concolic testing engine called CSF for heap-manipulating programs based on separation logic. CSF effectively combines specification-based testing and concolic execution for test input generation. It is evaluated on a set of challenging heapmanipulating programs. The results show that CSF generates valid test inputs with high coverage efficiently. Furthermore, we show that CSF can be potentially used in combination with precondition inference tools to reduce the user effort.

show abstract

“…So far, a substantial research effort in the area has targeted structural coverage [15,[28][29][30][31][32][33][34][35]. In particular, we briefly survey the techniques that investigated combinations of SBST and symbolic execution [16,36,37]. Other researchers exploit symbolic execution to optimize the evolutionary algorithms used in SBST [38][39][40][41].…”

Section: Related Workmentioning

confidence: 99%

Facilitating program performance profiling via evolutionary symbolic execution

Aquino

Braione

Denaro

et al. 2019

Software Testing Verif & Rel

Self Cite

View full text Add to dashboard Cite

Summary Performance profiling can benefit from test cases that hit high‐cost executions of programs. In this paper, we investigate the problem of automatically generating test cases that trigger the worst‐case execution of programs and propose a novel technique that solves this problem with an unprecedented combination of symbolic execution and evolutionary algorithms. Our technique, which we refer to as ‘Evolutionary Symbolic Execution’, embraces the execution cost of the program paths as the fitness function to pursue the worst execution. It defines an original set of evolutionary operators, based on symbolic execution, which suitably sample the possible program paths to make the search process effective. Specifically, our technique defines a memetic algorithm that (i) incrementally evolves by steering symbolic execution to traverse new program paths that comply with execution conditions combined and refined from the currently collected worse program paths and (ii) periodically applies local optimizations to the execution conditions of the worst currently identified program path to further speed up the identification of the worst path. We report on a set of initial experiments indicating that our technique succeeds in generating good worst‐case test cases for programs with which existing approaches cannot cope. Also, we show that, as far as the problem of generating worst‐case test cases is concerned, the distinguishing evolutionary operators based on symbolic execution that we define in this paper are more effective than traditional operators that directly manipulate the program inputs.

show abstract

Combining symbolic execution and search-based testing for programs with complex heap inputs

Cited by 55 publications

References 53 publications

A benchmark-based evaluation of search-based crash reproduction

A benchmark-based evaluation of search-based crash reproduction

Concolic Testing Heap-Manipulating Programs

Facilitating program performance profiling via evolutionary symbolic execution

Contact Info

Product

Resources

About