Evaluating the Effects of Compiler Optimizations on Mutation Testing at the Compiler IR Level

Hariri, Farah; Shi, August; Converse, Hayes; Khurshid, Sarfraz; Marinov, Darko

doi:10.1109/issre.2016.51

Cited by 15 publications

(11 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Coles et al adopt such an approach for mutating Java bytecode [147]. Derezinska and Kowalski [8] and Hariri et al [148] adopt the same approach for mutating the Common Intermediate Language of .NET and LLVM Bitcode, respectively.…”

Section: Mutant Creation (Step 2)mentioning

confidence: 99%

Mutation Testing Advances: An Analysis and Survey

Papadakis

Kintis

Zhang

et al. 2019

Advances in Computers

357

343

View full text Add to dashboard Cite

Mutation testing realises the idea of using artificial defects to support testing activities. Mutation is typically used as a way to evaluate the adequacy of test suites, to guide the generation of test cases and to support experimentation.Mutation has reached a maturity phase and gradually gains popularity both in academia and in industry. This chapter presents a survey of recent advances, over the past decade, related to the fundamental problems of mutation testing and sets out the challenges and open problems for the future development of the method. It also collects advices on best practices related to the use of mutation in empirical studies of software testing. Thus, giving the reader a 'mini-handbook'-style roadmap for the application of mutation testing as experimental methodology.

show abstract

Section: Mutant Creation (Step 2)mentioning

confidence: 99%

Mutation Testing Advances: An Analysis and Survey

Papadakis

Kintis

Zhang

et al. 2019

Advances in Computers

357

343

View full text Add to dashboard Cite

show abstract

“…Thus, a mutation operator is defined as pair of original instruction type and mutated instruction type. The To reduce the influence of redundant and equivalent mutants, we applied TCE (Papadakis et al 2015;Hariri et al 2016;Kintis et al 2018). Since we operate on LLVM bitcode we compared the mutated optimized LLVM codes using the llvm-diff utility.…”

Section: Automated Tools Usedmentioning

confidence: 99%

Selecting fault revealing mutants

et al. 2019

View full text Add to dashboard Cite

Mutant selection refers to the problem of choosing, among a large number of mutants, the (few) ones that should be used by the testers. In view of this, we investigate the problem of selecting the fault revealing mutants, i.e., the mutants that are killable and lead to test cases that uncover unknown program faults. We formulate two variants of this problem: the fault revealing mutant selection and the fault revealing mutant prioritization. We argue and show that these problems can be tackled through a set of 'static' program features and propose a machine learning approach, named FaRM, that learns to select and rank killable and fault revealing mutants. Experimental results involving 1,692 real faults show the practical benefits of our approach in both examined problems. Our results show that FaRM achieves a good trade-off between application cost and effectiveness (measured in terms of faults revealed). We also show that FaRM outperforms all the existing mutant selection methods, i.e., the random mutant sampling, the selective mutation and defect prediction (mutating the code areas pointed by defect prediction). In particular, our results show that with respect to mutant selection, our approach reveals 23% to 34% more faults than any of the baseline methods, while, with respect to mutant prioritization, it achieves higher average percentage of revealed faults with a median difference between 4% and 9% (from the random mutant orderings).

show abstract

“…Our questions on the effect of code structure on OBC was inspired by Rajan et al [23] who investigated the effect of program structure on the efficacy of MC/DC. Our focus on the effect of compiler optimization is inspired by Hariri et al [34] and Bullseye's statement about the effect of compiler optimization on OBC on their website [8]. This work is also closely related to the empirical comparison performed by Shams and Edwards [35] in which they conclude that OBC is a practical coverage criterion that is comparable to MC/DC.…”

Section: B What Contributes To the Improvement Of Flag-use Obc?mentioning

confidence: 97%

Toward Rigorous Object-Code Coverage Criteria

Byun¹,

Sharma²,

Rayadurgam

et al. 2017

2017 IEEE 28th International Symposium on Software Reliability Engineering (ISSRE)

View full text Add to dashboard Cite

Object-branch coverage (OBC) is often used as a measure of the thoroughness of tests suites, augmenting or substituting source-code based structural criteria such as branch coverage and modified condition/decision coverage (MC/DC). In addition, with the increasing use of third-party components for which source-code access may be unavailable, robust object-code coverage criteria are essential to assess how well the components are exercised during testing. While OBC has the advantage of being programming language independent and is amenable to non-intrusive coverage measurement techniques, variations in compilers, and the optimizations they perform can substantially change what is seen as an object branch, which itself appears to be an informally understood concept. To address the need for a robust object coverage criterion, this paper proposes a rigorous definition of OBC such that it captures well the semantic of source code branches for a given instruction set architecture. We report an empirical assessment of these criteria for the Intel x86 instruction set on several examples from embedded control systems software. Preliminary results indicate that object-code coverage can be made robust to compilation variations and is comparable in its bug-finding efficacy to source level MC/DC.

show abstract

Evaluating the Effects of Compiler Optimizations on Mutation Testing at the Compiler IR Level

Cited by 15 publications

References 24 publications

Mutation Testing Advances: An Analysis and Survey

Mutation Testing Advances: An Analysis and Survey

Selecting fault revealing mutants

Toward Rigorous Object-Code Coverage Criteria

Contact Info

Product

Resources

About