SmartCheck

Abstract

QuickCheck is a powerful library for automatic test-case generation. Because QuickCheck performs random testing, some of the counterexamples discovered are very large. QuickCheck provides an interface for the user to write shrink functions to attempt to reduce the size of counterexamples. Hand-written implementations of shrink can be complex, inefficient, and consist of significant boilerplate code. Furthermore, shrinking is only one aspect in debugging: counterexample generalization is the process of extrapolating from individual counterexamples to a class of counterexamples, often requiring a flash of insight from the programmer. To improve counterexample reduction and generalization, we introduce SmartCheck. SmartCheck is a debugging tool that reduces algebraic data using generic search heuristics to efficiently find smaller counterexamples. In addition to shrinking, SmartCheck also automatically generalizes counterexamples to formulas representing classes of counterexamples. SmartCheck has been implemented for Haskell and is freely available.

Abstracting failure-inducing inputs

“…This is an important information which was not visible in the minimized input fragment. In fact, this addresses a major drawback of current HDD variants as pointed out by Regehr et al [16] (generalized transformations) and Pike [15] (sharing).…”

“…We note that isolating independent causes is a well known problem for variants of delta debugging [15,16], and our algorithm provides a solution if applied directly to non-reduced input (under the constraint that the faults can be isolated to separate parts of the input). That is, if the input to the algorithm contains two separate faults, both faults will be concretized and retained in the output.…”

“…Another notable approach close to our own is universal sub-value generalization by Lee Pike [15]. The approach by Pike shows how one can use a forall construct in Haskell to indicate generalizable constructors in the Haskell representation of an AST.…”