A bug reproduction approach based on directed model checking and crash traces

Nayrolles, Mathieu; Hamou-Lhadj, Abdelwahab; Tahar, Sofiène; Larsson, Alf

doi:10.1002/smr.1789

Cited by 19 publications

(40 citation statements)

References 32 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…While BugRedux [14] is based on forward symbolic execution, Star [4] is a more recent approach that applies optimized backward symbolic execution on the method calls recorded in a stack trace in order to compute the input parameters that trigger the target crash. JCharming [19] is also based on using crash stack traces as the only source of information about a reported crash.…”

Section: Related Workmentioning

confidence: 99%

“…JCharming [19] applies directed model checking to identify the pre-conditions and input parameters that cause the target crash. Finally, ConCrash [3] is a recent approach that focuses on reproducing concurrency crashes, in particular.…”

Section: Related Workmentioning

confidence: 99%

“…Crash reproduction is an important step in debugging field crashes. Therefore, various automated approaches to crash reproduction [3,4,19,20,22,24] have been proposed in the literature. Among these, EvoCrash [22] is a search-based approach, which applies a Guided Genetic Algorithm (GGA) to generate a crashreproducing test.…”

Section: Introductionmentioning

confidence: 99%

“…Soltani et. al [22] evaluated EvoCrash on 50 real-world crashes and showed that the search-based approach improved over other non-search-based approaches proposed in the related literature [4,19,24].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction

Soltani

Derakhshanfar

Panichella

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

EvoCrash is a recent search-based approach to generate a test case that reproduces reported crashes. The search is guided by a fitness function that uses a weighted sum scalarization to combine three different heuristics: (i) code coverage, (ii) crash coverage and (iii) stack trace similarity. In this study, we propose and investigate two alternatives to the weighted sum scalarization: (i) the simple sum scalarization and (ii) the multi-objectivization, which decomposes the fitness function into several optimization objectives as an attempt to increase test case diversity. We implemented the three alternative optimizations as an extension of EvoSuite, a popular search-based unit test generator, and applied them on 33 real-world crashes. Our results indicate that for complex crashes the weighted sum reduces the test case generation time, compared to the simple sum, while for simpler crashes the effect is the opposite. Similarly, for complex crashes, multi-objectivization reduces test generation time compared to optimizing with the weighted sum; we also observe one crash that can be replicated only by multi-objectivization. Through our manual analysis, we found out that when optimizing the original weighted function gets trapped in local optima, optimization for decomposed objectives improves the search for crash reproduction. Generally, while multi-objectivization is under-explored, our results are promising and encourage further investigations of the approach.Partially funded by the EU Horizon 2020 ICT-10-2016-RIA "STAMP" project (No.731529) and the Dutch 4TU project "Big Software on the Run".

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction

Soltani

Derakhshanfar

Panichella

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

show abstract

“…To help developers in this process, various automated techniques have been suggested. These techniques typically either use program runtime data (Artzi et al 2008;Clause and Orso 2007;Narayanasamy et al 2005;Steven et al 2000;Gómez et al 2016;Bell et al 2013;Cao et al 2014;Rößler et al 2013) or crash stack traces (Bianchi et al 2017;Soltani et al 2017;Nayrolles et al 2017;Xuan et al 2015;Chen and Kim 2015) to generate a test case that triggers the reported crash.…”

mentioning

confidence: 99%

A benchmark-based evaluation of search-based crash reproduction

et al. 2019

View full text Add to dashboard Cite

Crash reproduction approaches help developers during debugging by generating a test case that reproduces a given crash. Several solutions have been proposed to automate this task. However, the proposed solutions have been evaluated on a limited number of projects, making comparison difficult. In this paper, we enhance this line of research by proposing JCrashPack, an extensible benchmark for Java crash reproduction, together with ExRunner, a tool to simply and systematically run evaluations. JCrashPack contains 200 stack traces from various Java projects, including industrial open source ones, on which we run an extensive evaluation of EvoCrash, the state-of-the-art tool for search-based crash reproduction. EvoCrash successfully reproduced 43% of the crashes. Furthermore, we observed that reproducing NullPointerException, IllegalArgumentException, and IllegalStateException is relatively easier than reproducing ClassCastException, ArrayIndexOutOfBoundsException and StringIndexOutOfBoundsException. Our results include a detailed manual analysis of EvoCrash outputs, from which we derive 14 current challenges for crash reproduction, among which the generation of input data and the handling of abstract and anonymous classes are the most frequents. Finally, based on those challenges, we discuss future research directions for search-based crash reproduction for Java.

show abstract

Search‐based crash reproduction using behavioural model seeding

Derakhshanfar

Devroey

Perrouin

et al. 2020

Software Testing Verif & Rel

View full text Add to dashboard Cite

Summary Search‐based crash reproduction approaches assist developers during debugging by generating a test case, which reproduces a crash given its stack trace. One of the fundamental steps of this approach is creating objects needed to trigger the crash. One way to overcome this limitation is seeding: using information about the application during the search process. With seeding, the existing usages of classes can be used in the search process to produce realistic sequences of method calls, which create the required objects. In this study, we introduce behavioural model seeding: a new seeding method that learns class usages from both the system under test and existing test cases. Learned usages are then synthesized in a behavioural model (state machine). Then, this model serves to guide the evolutionary process. To assess behavioural model seeding, we evaluate it against test seeding (the state‐of‐the‐art technique for seeding realistic objects) and no seeding (without seeding any class usage). For this evaluation, we use a benchmark of 122 hard‐to‐reproduce crashes stemming from six open‐source projects. Our results indicate that behavioural model seeding outperforms both test seeding and no seeding by a minimum of 6% without any notable negative impact on efficiency.

show abstract

A bug reproduction approach based on directed model checking and crash traces

Cited by 19 publications

References 32 publications

Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction

Single-objective Versus Multi-objectivized Optimization for Evolutionary Crash Reproduction

A benchmark-based evaluation of search-based crash reproduction

Search‐based crash reproduction using behavioural model seeding

Contact Info

Product

Resources

About