Test-case reduction for C compiler bugs

Regehr, John; Chen, Yang; Cuoq, Pascal; Eide, Eric; Ellison, Chucky; Yang, Xuejun

doi:10.1145/2254064.2254104

Cited by 198 publications

(35 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our approach combines the speed of delta debugging with the power of QuickCheck to randomly discover structurally smaller counterexamples. The idea of randomization in test-case reduction was independently developed at approximately the same time as SmartCheck and first published in the domain of reducing C programs that demonstrate compiler bugs [17]. We believe our work is the first to explore the idea of counterexample generalization.…”

Section: Related Workmentioning

confidence: 95%

SmartCheck

Pike

2014

Proceedings of the 2014 ACM SIGPLAN Symposium on Haskell

View full text Add to dashboard Cite

QuickCheck is a powerful library for automatic test-case generation. Because QuickCheck performs random testing, some of the counterexamples discovered are very large. QuickCheck provides an interface for the user to write shrink functions to attempt to reduce the size of counterexamples. Hand-written implementations of shrink can be complex, inefficient, and consist of significant boilerplate code. Furthermore, shrinking is only one aspect in debugging: counterexample generalization is the process of extrapolating from individual counterexamples to a class of counterexamples, often requiring a flash of insight from the programmer. To improve counterexample reduction and generalization, we introduce SmartCheck. SmartCheck is a debugging tool that reduces algebraic data using generic search heuristics to efficiently find smaller counterexamples. In addition to shrinking, SmartCheck also automatically generalizes counterexamples to formulas representing classes of counterexamples. SmartCheck has been implemented for Haskell and is freely available.

show abstract

Section: Related Workmentioning

confidence: 95%

SmartCheck

Pike

2014

Proceedings of the 2014 ACM SIGPLAN Symposium on Haskell

View full text Add to dashboard Cite

show abstract

“…At a high-level, delta-debugging works by gradually reducing the input program and ensuring that the reduced program still triggers the bug and is valid (i.e., does not contain undefined behavior). State-of-the art delta-debugging reducers include Berkeley Delta [13] and C-Reduce [19]. Unfortunately, these reducers support reducing only a single file.…”

Section: Bug Reductionmentioning

confidence: 99%

“…CCG is a random C program generator that focuses on finding compiler crashing bugs [3]. Csmith is another C program generator that can find both crashing and miscompilation bugs [4,19,25]. Based on the idea of differential testing [12], Csmith randomly generates C programs and checks for deviant behavior across compilers or compiler versions.…”

Section: Related Workmentioning

confidence: 99%

Randomized stress-testing of link-time optimizers

Sun

2015

Proceedings of the 2015 International Symposium on Software Testing and Analysis

View full text Add to dashboard Cite

Link-time optimization (LTO) is an increasingly important and adopted modern optimization technology. It is currently supported by many production compilers, including GCC, LLVM, and Microsoft Visual C/C++. Despite its complexity, but because it is more recent, LTO is relatively less tested compared to the more mature, traditional optimizations. To evaluate and help improve the quality of LTO, we present the first extensive effort to stress-test the LTO components of GCC and LLVM, the two most widely-used production C compilers. In 11 months, we have discovered and reported 37 bugs (12 in GCC; 25 in LLVM). Developers have confirmed 21 of our bugs, and fixed 11 of them.Our core technique is differential testing and realized in the tool Proteus. We leverage existing compiler testing tools (Csmith and Orion) to generate single-file test programs and address two important challenges specific for LTO testing. First, to thoroughly exercise LTO, Proteus automatically transforms a single-file program into multiple compilation units and stochastically assigns each an optimization level. Second, for effective bug reporting, we develop a practical mechanism to reduce LTO bugs involving multiple files. Our results clearly demonstrate Proteus's utility; we plan to make ours a continuous effort in validating link-time optimizers.

show abstract

“…Regehr et al [13] have suggested to generalize test-case minimization problem as a generic search problem which can be tuned with various fitness functions. They have introduced three new reducers and applied their framework for the reduction of C compiler bugs.…”

Section: Related Workmentioning

confidence: 99%

Reduce first, debug later

Elyasov

Prasetya

Hage

et al. 2014

Proceedings of the 9th International Workshop on Automation of Software Test

View full text Add to dashboard Cite

The delta debugging minimization algorithm (ddmin) provides an efficient procedure for the simplification of failing test-cases. Despite its contribution towards the automation of debugging, ddmin still requires a significant number of iterations to complete. The delta debugging (DD) search space can be narrowed down by providing the test-case circumstances that are most likely relevant to the occurred failure. This paper proposes a novel approach to the problem of failure simplification consisting of two consecutive phases: 1) failure reduction by rewriting (performed offline), and 2) DD invocation (performed online). In the best case scenario, the reduction phase may already deliver a simplified failure, otherwise, it potentially supplies DD with extra information about where to look for the failure. The proposed solution has been prototyped as a web application debugging tool, which was evaluated on a shopping cart web application -Flex Store. The evaluation shows an improvement of the DD execution time if the offline reduction over-approximates the failure.

show abstract

Test-case reduction for C compiler bugs

Cited by 198 publications

References 15 publications

SmartCheck

SmartCheck

Randomized stress-testing of link-time optimizers

Reduce first, debug later

Contact Info

Product

Resources

About