Dissecting American Fuzzy Lop: A FuzzBench Evaluation

Abstract

AFL is one of the most used and extended fuzzer, adopted by industry and academic researchers alike. While the community agrees on AFL’s effectiveness at discovering new vulnerabilities and at its outstanding usability, many of its internal design choices remain untested to date. Security practitioners often clone the project “as-is” and use it as a starting point to develop new techniques, usually taking everything under the hood for granted. Instead, we believe that a careful analysis of the different parameters could help modern fuzzers to improve their performance and explain how each choice can affect the outcome of security testing, either negatively or positively. The goal of this paper is to provide a comprehensive understanding of the internal mechanisms of AFL by performing experiments and by comparing different metrics used to evaluate fuzzers. This can help to show the effectiveness of some techniques and to clarify which aspects are instead outdated. To perform our study we performed nine unique experiments that we carried out on the popular Fuzzbench platform. Each test focuses on a different aspect of AFL, ranging from its mutation approach to the feedback encoding scheme and its scheduling methodologies. Our findings show that each design choice affects different factors of AFL. While some of these are positively correlated with the number of detected bugs or the coverage of the target application, other features are instead related to usability and reliability. Most important, we believe that the outcome of our experiments indicates which parts of AFL we should preserve in the design of modern fuzzers.

Predictive Context-sensitive Fuzzing

“…FuzzBench targets realworld programs, pinning specific versions for reproducibility and result validation [17]. We select the 'type: bug' configuration of FuzzBench, a choice made also in other recent bug-oriented studies [49], [50], [51]. We study different dimensions of our approach for the following research questions:…”

A multilevel comprehensive fuzzy evaluation algorithm for highway route schemes considering operating speed and association rules

“…Limit the priority score ∋ ( p S of the decision objective to between 1 and 10, and match the corresponding weights based on the importance of the evaluation factor attributes of the highway route plan. Map the influence index 18 ÃA of the criterion layer to the priority score of the decision-maker, and add input index attributes through the inference module of the model to obtain the membership function and fuzzy rules corresponding to each language value [5]. After mapping, obtain the fuzzy calculation result.…”

CrabSandwich: Fuzzing Rust with Rust (Registered Report)

“…Graybox fuzzing [17,28,45], on the other hand, uses only lightweight code instrumentationusually to trace code coverage -to produce feedback that is used to evaluate the quality of a test case that is kept for further mutations if "interesting". This approach has become the leading technique to discover vulnerabilities in modern codebases, thanks to the popularity of AFL [19,51] and Google's OSS-Fuzz [1] being a prime example of a large-scale deployment of graybox fuzzing.…”