Feat

Abstract

In mathematics, an enumeration of a set S is a bijective function from (an initial segment of) the natural numbers to S. We define "functional enumerations" as efficiently computable such bijections. This paper describes a theory of functional enumeration and provides an algebra of enumerations closed under sums, products, guarded recursion and bijections. We partition each enumerated set into numbered, finite subsets.We provide a generic enumeration such that the number of each part corresponds to the size of its values (measured in the number of constructors). We implement our ideas in a Haskell library called testing-feat, and make the source code freely available. Feat provides efficient "random access" to enumerated values. The primary application is property-based testing, where it is used to define both random sampling (for example QuickCheck generators) and exhaustive enumeration (in the style of SmallCheck). We claim that functional enumeration is the best option for automatically generating test cases from large groups of mutually recursive syntax tree types. As a case study we use Feat to test the prettyprinter of the Template Haskell library (uncovering several bugs).

Property-Based Testing via Proof Reconstruction

“…A better approach is to introduce an FPC for a form of iterative deepening for exact bounds, where we output only those candidates requiring that precise bound. This has some similarity with the approach in Feat [19]. The interested reader can peruse this FPC in the code accompanying our paper.…”

“…This tumultuous rate of growth is characterized, in our opinion, by a lack of common (logical) foundation. For one, PBT comes in different flavors as far as data generation is concerned: while random generation is the most common one, other tools employ exhaustive generation ( [12,54]) or a combination thereof ( [19]). At the same time, one could say that PBT is rediscovering logic and, in particular, logic programming: to begin with, QuickCheck's DSL is basically Horn clause logic; LazySmallCheck [54] has adopted narrowing to permit less redundant testing over partial rather than ground terms; PLT-Redex [20] contains a re-implementation of constraint logic programming in order to better generate well-typed λ-terms [22].…”

Describing Console I/O Behavior for Testing Student Submissions in Haskell

“…But in general, such conditions can be far more complicated. A possible solution to this problem might be to not rely on random test case generation and shrinking in the first place but instead use an enumerative testing approach [2,6], systematically generating test cases up to a certain size. However, we currently have not done any concrete work in this direction.…”

“…Following the approach presented by Swierstra and Altenkirch [8], corresponding tests can be automated. First, an alternative monad is defined that represents a semantic domain for console I/O programs 2 : Now, any potential Haskell solution to the I/O task given further above, main = do . .…”

Julia subtyping: a rational reconstruction

“…The key idea is to randomly generate one term, and derive a second term by mutation of the first one. Our fuzzer relies on the FEAT library by Duregård et al [2012] to enumerate exhaustively up to a 113:17 certain size pre-types over a simplified algebra of type constructors:…”