Value Prediction and Speculative Execution on GPU

Abstract

GPUs and CPUs have fundamentally different architectures. It is conventional wisdom that GPUs can accelerate only those applications that exhibit very high parallelism, especially vector parallelism such as image processing. In this paper, we explore the possibility of using GPUs for value prediction and speculative execution: we implement software value prediction techniques to accelerate programs with limited parallelism, and software speculation techniques to accelerate programs that contain runtime parallelism, which are hard to parallelize statically. Our experiment results show that due to the relatively high overhead, mapping software value prediction techniques on existing GPUs may not bring any immediate performance gain. On the other hand, although software speculation techniques introduce some overhead as well, mapping these techniques to existing GPUs can already bring some performance gain over CPU. Based on these observations, we explore the hardware implementation of speculative execution operations on GPU architectures to reduce the software performance overheads. The results indicate that the hardware extensions result in almost tenfold reduction of the control divergent sequential operations with only moderate hardware (5-8%) and power consumption (1-5%) overheads.

The Performance Optimization of Threaded Prefetching for Linked Data Structures

“…With the advent of Chip multiprocessors (CMP) architectures, thread-based prefetching and speculative execution techniques have received much attention in the research community [9][10][11][12][13][14][15][16][17]. Although CMP suffers greater than ever before from memory latencies problems, it also brings new opportunities for data prefetching due to the fact that it can execute multiple threads in parallel.…”

Optimistic Parallelism on GPUs

“…Unlike their work, this paper explores thread-level speculative parallelism in a kernel. An exploratory study has been done for speculative execution on GPUs [10,11]. They explored the hardware implementation of speculative execution operations on GPU architectures to reduce the software performance overheads.…”

Leveraging GPUs using cooperative loop speculation

“…While none of the previous works on automatic compilation for current GPUs considered speculation, there are other works [Menon et al 2012;Diamos and Yalamanchili 2010;Liu et al 2011] that studied the possibility of speculative execution on the GPU. Menon et al [2012] modified the GPU hardware to support voltage speculation.…”

“…This approach leveraged the possibility of speculatively partitioning several kernels on multiple GPUs. Liu et al [2011] showed the possibility of using GPUs for speculative execution using a GPU-like architecture on FPGAs. They implemented software value prediction techniques to accelerate programs with limited parallelism, and software speculation techniques, which re-executes the whole loop in case of a dependency violation.…”