Zorua: A holistic approach to resource virtualization in GPUs

Vijaykumar, Nandita; Hsieh, Kevin; Pekhimenko, Gennady; Khan, Samira; Shrestha, Ashish; Ghose, Saugata; Jog, Adwait; Gibbons, Phillip B.; Mutlu, Onur

doi:10.1109/micro.2016.7783718

Cited by 49 publications

(38 citation statements)

References 62 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…No CTAs will be dispatched to an SM if one of these resources is insufficient to support a new CTA. We now describe these TLP-related structures in more detail, as previously detailed in the literature [5], [29], [44], [52]. Register file: The maximum number of concurrent threads per SM is a function of register file capacity on the one hand, and the number of registers allocated per thread on the other hand.…”

Section: Tlp-related Hardware Structuresmentioning

confidence: 99%

“…Warp slots: The number of warp slots is the third resource that may limit the maximum number of CTAs per SM [44], [52]. As mentioned before, a warp is the basic unit to schedule and issue instructions on a GPU.…”

Section: Tlp-related Hardware Structuresmentioning

confidence: 99%

“…This works because GPU kernels are made up of many CTAs that exhibit relatively consistent behavior, so we can use some initial CTAs to help us guide scheduling for future CTAs [27]. Although finegrained phase behavior may be observed at the warp level [21], [44], this gets leveled out at the SM level as several CTAs execute concurrently on an SM. Finally, new CUDA features such as dynamic parallelism in which parent kernels can launch child kernels to run alongside the parent kernels, may affect kernel launch.…”

Section: Ipc Basementioning

confidence: 99%

“…To increase TLP without incurring extra hardware, Yoon et al [52] propose the Virtual Thread (VT) architecture: by storing the context of inactive CTAs in shared memory, VT enables assigning more CTAs to an SM. Vijaykumar et al [44] introduce Zorua, a resource virtualization framework for on-chip TLP-related resources. On the other hand, Kayıran et al [23] and Lee et al [28] make the observation that performance decreases with an increasing number of CTAs per SM for some kernels, and hence modulate TLP by allocating the optimal number of CTAs per SM to improve performance.…”

Section: Related Workmentioning

confidence: 99%

See 3 more Smart Citations

HeteroCore GPU to Exploit TLP-Resource Diversity

Zhao

Wang

Eeckhout

2019

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

Graphics processing units (GPUs) are widely adopted as compute accelerators in cloud computing environments and supercomputers. Sharing GPU resources in such environments requires effective multitasking support. Unfortunately, conventional GPUs lack the ability to adapt to diverse thread-level parallelism (TLP) resource demands among co-executing kernels. Previous work such as SM partitioning and simultaneously multitasking (SMK) increase system throughput, however, they degrade per-application performance significantly. This paper proposes the HeteroCore GPU to significantly improve multitasking performance with a similar area cost as a conventional GPU. After rebalancing TLP-related SM resources, a HeteroCore GPU consists of two types of SMs to support diverse TLP-resource demands. Dynamic scheduling performs low-overhead spatial profiling during runtime across the different SM types and steers scheduling decisions based on the TLP-resource demands of the co-executing kernels. Compared to a conventional GPU, HeteroCore GPU improves system throughput by 20.1% on average (up to 80.9%) and per-application performance by 29.8% on average (up to 50.3%), for workload mixes composed of kernels with different TLP-resource demands.

show abstract

Section: Tlp-related Hardware Structuresmentioning

confidence: 99%

Section: Tlp-related Hardware Structuresmentioning

confidence: 99%

Section: Ipc Basementioning

confidence: 99%

Section: Related Workmentioning

confidence: 99%

See 2 more Smart Citations

HeteroCore GPU to Exploit TLP-Resource Diversity

Zhao

Wang

Eeckhout

2019

IEEE Trans. Parallel Distrib. Syst.

View full text Add to dashboard Cite

show abstract

“…The initial workload characterization phase works because although GPU applications exhibit phase behavior at the warp level [18,19], this gets leveled out as several TBs execute concurrently [20]. After the classification phase, CD-search chooses to enter the performance mode or the power mode.…”

Section: Classification-driven Searchmentioning

confidence: 99%

Classification-Driven Search for Effective SM Partitioning in Multitasking GPUs

Zhao

Wang

Eeckhout

2018

Proceedings of the 2018 International Conference on Supercomputing

View full text Add to dashboard Cite

Graphics processing units (GPUs) feature an increasing number of streaming multiprocessors (SMs) with each successive generation. At the same time, GPUs are increasingly widely adopted in cloud services and data centers to accelerate general-purpose workloads. Running multiple applications on a GPU in such environments requires effective multitasking support. Spatial multitasking in which independent applications co-execute on different sets of SMs is a promising solution to share GPU resources. Unfortunately, how to effectively partition SMs is an open problem. In this paper, we observe that compared to widely-used even partitioning, dynamic SM partitioning based on the characteristics of the co-executing applications can significantly improve performance and power efficiency. Unfortunately, finding an effective SM partition is challenging because the number of possible combinations increases exponentially with the number of SMs and co-executing applications. Through offline analysis, we find that first classifying workloads, and then searching an effective SM partition based on the workload characteristics can significantly reduce the search space, making dynamic SM partitioning tractable. Based on these insights, we propose Classification-Driven search (CD-search) for low-overhead dynamic SM partitioning in multitasking GPUs. CD-search first classifies workloads using a novel off-SM bandwidth model, after which it enters the performance mode or power mode depending on the workload's characteristics. Both modes follow a specific search strategy to quickly determine the optimum SM partition. Our evaluation shows that CD-search improves system throughput by 10.4% on average (and up to 62.9%) over even partitioning for workloads that are classified for the performance mode. For workloads classified for the power mode, CD-search reduces power consumption by 25% on average (and up to 41.2%). CD-search incurs limited runtime overhead. CCS CONCEPTS • Computer systems organization → Single instruction, multiple data;

show abstract