Efficient GPU Spatial-Temporal Multitasking

Liang, Yun; Huynh, Huynh Phung; Rupnow, Kyle; Goh, Rick Siow Mong; Chen, Deming

doi:10.1109/tpds.2014.2313342

Cited by 80 publications

(23 citation statements)

References 12 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…11(b)(2)). Several software [7,9,11] and hardware [1,12,2] approaches for GPU multitasking has been introduced before. These works generally focus on SM level partitioning, and did not consider resources inside SMs.…”

Section: Case Study: Pf+bp and Hs+smmentioning

confidence: 99%

Efficient GPU multitasking with latency minimization and cache boosting

Kim

Chu

Park

2017

IEICE Electron. Express

View full text Add to dashboard Cite

GPU spatial multitasking has been proven to be quite effective at executing different applications concurrently using SM partitioning. However, while it maximizes total throughput, latency-critical applications often cannot meet their deadlines due to the increased execution time. Furthermore, SM partitioning cannot allocate the appropriate L1 cache size per kernel. To solve these problems, this paper proposes a new application-aware resource allocation framework called GPU Fine-Tuner, for assigning appropriate resources to GPU kernels. To minimize the execution time of latencyconstrained applications, it assigns them more SMs when performance is not affected. It also increases the cache size of SMs for cache-sensitive kernels using resource borrowing from neighbors for cache-insensitive kernels. Experimental results show that the Fine-Tuner outperforms GPU spatial multitasking with up to 15% less average latency without performance degradation.

show abstract

Section: Case Study: Pf+bp and Hs+smmentioning

confidence: 99%

Efficient GPU multitasking with latency minimization and cache boosting

Kim

Chu

Park

2017

IEICE Electron. Express

View full text Add to dashboard Cite

show abstract

“…Optimization techniques have also been proposed to tailor the applications to the underlying architectural features and improve the resource utilization. The state-of-the-art GPU performance optimization techniques focused on thread and warp scheduling, cache optimization, register allocation optimization, power and aging optimization, multitasking,etc [25]- [32].…”

Section: Related Workmentioning

confidence: 99%

An Accurate GPU Performance Model for Effective Control Flow Divergence Optimization

Liang

Satria

Rupnow

et al. 2016

IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst.

Self Cite

View full text Add to dashboard Cite

Graphic processing units (GPUs) are composed of a group of single-instruction multiple data (SIMD) streaming multiprocessors (SMs). GPUs are able to efficiently execute highly data parallel tasks through SIMD execution on the SMs. However, if those threads take diverging control paths, all divergent paths are executed serially. In the worst case, every thread takes a different control path and the highly parallel architecture is used serially by each thread. This control flow divergence problem is well known in GPU development; code transformation, memory access redirection, and data layout reorganization are commonly used to reduce the impact of divergence. These techniques attempt to eliminate divergence by grouping together threads or data to ensure identical behavior. However, prior efforts using these techniques do not model the performance impact of any particular divergence or consider that complete elimination of divergence may not be possible. Thus, we perform analysis of the performance impact of divergence and potential thread regrouping algorithms that eliminate divergence or minimize the impact of remaining divergence. Finally, we develop a divergence optimization framework that analyzes and transforms the kernel at compile-time and regroups the threads at run-time. For the compute-bound applications, our proposed metrics achieve performance estimation accuracy within 6.2% of measured performance. Using these metrics, we develop thread regrouping algorithms, which consider the impact of divergence, and speed up these applications by 2.2X on average on NVIDIA GTX480.

show abstract

“…Zhong et al [16] propose an algorithm to combine two suitable kernels with optimized slicing length to improve the GPU throughput. More recently, Liang et al [12] utilize concurrent kernel execution to achieve GPU spatial-temporal multitasking.…”

Section: Related Workmentioning

confidence: 99%

Improving GPGPU energy-efficiency through concurrent kernel execution and DVFS

Jiao¹,

Lu²,

Huynh³

et al. 2015

2015 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

Self Cite

View full text Add to dashboard Cite

Current generation GPUs can accelerate high-performance, computeintensive applications by exploiting massive thread-level parallelism. The high performance, however, comes at the cost of increased power consumption. Recently, commercial GPGPU architectures have introduced support for concurrent kernel execution to better utilize the computational/memory resources and thereby improve overall throughput. In this paper, we argue and experimentally validate the benefits of concurrent kernels towards energyefficient execution. We design power-performance models to carefully select the appropriate kernel combinations to be executed concurrently, the relative contributions of the kernels to the thread mix, along with the frequency choices for the cores and the memory to achieve high performance per watt metric. Our experimental evaluation shows that the concurrent kernel execution in combination with DVFS can improve energy-efficiency by up to 34.5% compared to the most energy-efficient sequential execution.

show abstract

Efficient GPU Spatial-Temporal Multitasking

Cited by 80 publications

References 12 publications

Efficient GPU multitasking with latency minimization and cache boosting

Efficient GPU multitasking with latency minimization and cache boosting

An Accurate GPU Performance Model for Effective Control Flow Divergence Optimization

Improving GPGPU energy-efficiency through concurrent kernel execution and DVFS

Contact Info

Product

Resources

About