SIMD divergence optimization through intra-warp compaction

Abstract

SIMD execution units in GPUs are increasingly used for high performance and energy efficient acceleration of general purpose applications. However, SIMD control flow divergence effects can result in reduced execution efficiency in a class of GPGPU applications, classified as divergent applications. Improving SIMD efficiency, therefore, has the potential to bring significant performance and energy benefits to a wide range of such data parallel applications.Recently, the SIMD divergence problem has received increased attention, and several micro-architectural techniques have been proposed to address various aspects of this problem. However, these techniques are often quite complex and, therefore, unlikely candidates for practical implementation. In this paper, we propose two micro-architectural optimizations for GPGPU architectures, which utilize relatively simple execution cycle compression techniques when certain groups of turned-off lanes exist in the instruction stream. We refer to these optimizations as basic cycle compression (BCC) and swizzled-cycle compression (SCC), respectively. In this paper, we will outline the additional requirements for implementing these optimizations in the context of the studied GPGPU architecture. Our evaluations with divergent SIMD workloads from OpenCL (GPGPU) and OpenGL (graphics) applications show that BCC and SCC reduce execution cycles in divergent applications by as much as 42% (20% on average). For a subset of divergent workloads, the execution time is reduced by an average of 7% for today's GPUs or by 18% for future GPUs with a better provisioned memory subsystem. The key contribution of our work is in simplifying the micro-architecture for delivering divergence optimizations while providing the bulk of the benefits of more complex approaches.

Compiler-Assisted Compaction/Restoration of SIMD Instructions

“…The main issue with predicated execution in SIMD architectures is its low energy efficiency. Measured mask density 2 is between 18-20% on typical benchmarks [23], [24], [25]. This means that sparse predicated masks are common on modern codes.…”

“…The hard timeout policy is implicit in every scenario. In the x-axis the number of cycles for each timeout policy control flow divergence [23], [24] does. Moreover, Vaidya et al [24] also demonstrate that the true-value position inside the mask register leads to no variability in performance.…”

“…In the x-axis the number of cycles for each timeout policy control flow divergence [23], [24] does. Moreover, Vaidya et al [24] also demonstrate that the true-value position inside the mask register leads to no variability in performance. For the sake of clarity we will omit the combination possibilities of the true-value positions inside the mask.…”

“…Vaidya et al [24] propose two micro-architectural techniques to improve the performance of predicated instructions in GPUs. They rely on the fact that the VL is usually multiple of the number of hardware execution units (or ALU-width).…”

Design of a High Performance Vector Processor Based on RISIC-V Architecture

“…To solve the problem of low performance of parallel data calculations [12] in industrial power applications, it is proposed to add a Vector processor hardware implementation named VPU in the TS800, which can support adaptive controllers Reinforcement learning and learningbased [13] underlying algorithm requirements. This design can support FFT and IFT of 64~4096 point [14].…”