Automated test generation for OpenCL kernels using fuzzing and constraint solving

Abstract

Graphics Processing Units (GPUs) are massively parallel processors offering performance acceleration and energy efficiency unmatched by current processors (CPUs) in computers. These advantages along with recent advances in the programmability of GPUs have made them attractive for general-purpose computations. Despite the advances in programmability, GPU kernels are hard to code and analyse due to the high complexity of memory sharing patterns, striding patterns for memory accesses, implicit synchronisation, and combinatorial explosion of thread interleavings. Existing few techniques for testing GPU kernels use symbolic execution for test generation that incur a high overhead, have limited scalability and do not handle all data types.We propose a test generation technique for OpenCL kernels that combines mutation-based fuzzing and selective constraint solving with the goal of being fast, effective and scalable. Fuzz testing for GPU kernels has not been explored previously. Our approach for fuzz testing randomly mutates input kernel argument values with the goal of increasing branch coverage. When fuzz testing is unable to increase branch coverage with random mutations, we gather path constraints for uncovered branch conditions and invoke the Z3 constraint solver to generate tests for them.In addition to the test generator, we also present a schedule amplifier that simulates multiple work-group schedules, with which to execute each of the generated tests. The schedule amplifier is designed to help uncover inter work-group data races. We evaluate the effectiveness of the generated tests and schedule amplifier using 217 kernels from open source projects and industry standard benchmark suites measuring branch coverage and fault finding. We find our test generation technique achieves close to 100% coverage and mutation score for majority of the kernels. Overhead incurred in test generation is small (average of 0.8 seconds). We also confirmed our technique scales easily to large kernels, and can support all OpenCL data types, including complex data structures. CCS CONCEPTS• Software and its engineering → Software testing and debugging; Massively parallel systems.

A Generative and Mutational Approach for Synthesizing Bug-Exposing Test Cases to Guide Compiler Fuzzing

“…AFL [88] is the first coverage-guided testing framework, and it employs compile-time instrumentation and genetic algorithms to assist in generating random test cases for covering more code branches. The subsequent works [17,38,51,60,66,69,84] further improve code coverage for domain-specific testing by mutating the seed programs. Poloto et al [67] proposed an interpreter-guided unit testing solution on the JIT compiler.…”

Testing Smart Contracts

“…It then switches to whitebox fuzzing to gather path conditions for uncovered conditions. These constraints for uncovered regions are given to an SMT solver which returns inputs exercising feasible paths [10,39].…”

“…The approach for GF is not new, and has been performed extensively in other domains [4,24,26,39]. There is no existing tool for smart contracts that uses GF, and our implementation is the first in this domain.…”

Predicting Patch Correctness Based on the Similarity of Failing Test Cases

“…In future work, the community could further investigate other datasets as well as test cases augmentation (through test generation [49,55,58] or code search [28]) to enlarge the datasets of patches and test cases. Threats to Internal Validity A major threat to internal validity is that we manually process patches to build the dataset.…”