Many-core compiler fuzzing

Abstract

We address the compiler correctness problem for many-core systems through novel applications of fuzz testing to OpenCL compilers. Focusing on two methods from prior work, random differential testing and testing via equivalence modulo inputs (EMI), we present several strategies for random generation of deterministic, communicating OpenCL kernels, and an injection mechanism that allows EMI testing to be applied to kernels that otherwise exhibit little or no dynamically-dead code. We use these methods to conduct a large, controlled testing campaign with respect to 21 OpenCL (device, compiler) configurations, covering a range of CPU, GPU, accelerator, FPGA and emulator implementations. Our study provides independent validation of claims in prior work related to the effectiveness of random differential testing and EMI testing, proposes novel methods for lifting these techniques to the many-core setting and reveals a significant number of OpenCL compiler bugs in commercial implementations.

Exposing errors related to weak memory in GPU applications

“…We see several avenues for future investigation: (a) the design of GPU weak memory-aware formal program analysis techniques (to enable verification of our empirically proposed fixes); (b) architectural investigation of the critical patches revealed by our micro-benchmarks, building on the insights into Nvidia GPUs provided by GPGPU-Sim [1]; (c) applying and evaluating our methods to CPU architectures and applications; (d) combining our techniques with recent methods for GPU compiler testing [29] in order to check the correctness of compilation in the presence of fine-grained concurrency; (e) combining our techniques with race detectors to help pinpoint communication idioms in applications and developing targeted testing around these locations.…”

Automatic testing of symbolic execution engines via program generation and differential testing

“…Our approach takes advantage of the recent work on testing compilers [22], [23], [34], especially the work of Regehr et al on program generation [34] and reduction [27]. Combined with our technique for generating program variants in three different modes and a set of appropriate oracles, we show that these techniques are effective at finding bugs in symbolic execution engines.…”

Metamorphic testing for (graphics) compilers

“…Because it will not be executed, arbitrary syntactically valid code can be injected, including code that would invoke undefined behaviour. In [5] we experimented with injecting code generated by a fuzzer. In our GLSL testing framework we instead consider transplanting code fragments from one real-world shader program into another (see Section 3).…”

“…[4,5,6,7,8]). A notable method for testing compilers is random differential testing, popularised by the Csmith tool [8], whereby a program is randomly generated (a process known as fuzzing) and then compiled by different compilers at multiple optimisation levels.…”

“…A metamorphic approach has also been used to test compilers by generating input programs that are equivalent by construction [7]. In recent work on testing compilers for the OpenCL manycore programming language, we experimented with a variation of EMI testing where instead of identifying existing Idead code, we injected I-dead code into programs [5]. This works by introducing a new program input variable, v say, and injecting conditional code of the form:…”