Just fuzz it: solving floating-point constraints using coverage-guided fuzzing

Abstract

We investigate the use of coverage-guided fuzzing as a means of proving satisfiability of SMT formulas over finite variable domains, with specific application to floating-point constraints. We show how an SMT formula can be encoded as a program containing a location that is reachable if and only if the program's input corresponds to a satisfying assignment to the formula. A coverage-guided fuzzer can then be used to search for an input that reaches the location, yielding a satisfying assignment. We have implemented this idea in a tool, Just Fuzz-it Solver (JFS), and we present a large experimental evaluation showing that JFS is both competitive with and complementary to state-of-the-art SMT solvers with respect to solving floating-point constraints, and that the coverage-guided approach of JFS provides significant benefit over naive fuzzing in the floating-point domain. Applied in a portfolio manner, the JFS approach thus has the potential to complement traditional SMT solvers for program analysis tasks that involve reasoning about floating-point constraints. CCS CONCEPTS • Theory of computation → Constraint and logic programming; • Software and its engineering → Software testing and debugging.

Fuzzing: A Survey for Roadmap

“…Thus, the constraint solver performs in a greybox manner, e.g., it utilizes linear functions to approximate constraint behaviors. Interestingly, researchers have paid attention to solving path constraints via fuzzing [24,108]. For example, JFS [108] translates SMT formulas to a program and utilizes coverage-guided fuzzing to explore the program.…”

“…Interestingly, researchers have paid attention to solving path constraints via fuzzing [24,108]. For example, JFS [108] translates SMT formulas to a program and utilizes coverage-guided fuzzing to explore the program. The SMT formulas are solved when a fuzzing-generated input reaches speciic locations of the corresponding program.…”

Fuzzing Symbolic Expressions

“…To show that Fu z z y -Sa t can be used in other frameworks, we integrate it also in QSYM [4]. In our experimental evaluation: 1) we compare Fu z z y -Sa t to the SMT solver Z3 [7] and the approximate solver JFS [8 ] on queries issued by QSYM, which we use as a mature baseline. o u r results suggest that Fu z z y -Sa t can provide a nice tradeoff between speed and solving effectiveness, i.e., the number of queries found satisfiable by a solver.…”

“…A different direction is instead taken by JFS [8 ], which builds on the experimental observation that SMT solvers can struggle on queries that involve floating-point values. JFS thus proposes to turn the query into a program, which is then analyzed using coverage-based grey-box fuzzing.…”

“…To further assess the effectiveness of Fu z z y -Sa t , we compare FUZZOLIC, which is built around this solver, against state-of-the-art binary open-source fuzzers on the 1 2 benchmarks, tracking the code coverage reached during 8 -hour experiments (10 runs). Besides FUZZOLIC, we consider: (a) AFL++ [2] rev.…”