Bandwidth-efficient on-chip interconnect designs for GPGPUs

Abstract

Modern computational workloads require abundant thread level parallelism (TLP), necessitating highly-parallel, manycore accelerators such as General Purpose Graphics Processing Units (GPGPUs). GPGPUs place a heavy demand on the on-chip interconnect between the many cores and a few memory controllers (MCs). Thus, traffic is highly asymmetric, impacting on-chip resource utilization and system performance. Here, we analyze the communication demands of typical GPGPU applications, and propose efficient Network-on-Chip (NoC) designs to meet those demands. We show that the proposed schemes improve performance by up to 64.7%. Compared to the best of class prior work, our VC monopolizing and partitioning schemes improve performance by 25%.

Memory-aware circuit overlay NoCs for latency optimized GPGPU architectures

“…We used GPGPU-Sim [19] to collect detailed application traces and simulated the network and memory traffic on a customized Noxim NoC simulator [21] that integrates our MACRO-NoC architecture model. We obtained traces for 11 CUDA benchmarks [8], [20], each with different number of kernels and levels of memory intensity: We compared our architecture with two prior works that also propose NoC architectures for GPGPUs: [10] and [11] (both are discussed in the Section 2). The architecture discussed in [10] is called Direct all-to-all (DA2), while that from [11] is called XY-YX.…”

“…We obtained traces for 11 CUDA benchmarks [8], [20], each with different number of kernels and levels of memory intensity: We compared our architecture with two prior works that also propose NoC architectures for GPGPUs: [10] and [11] (both are discussed in the Section 2). The architecture discussed in [10] is called Direct all-to-all (DA2), while that from [11] is called XY-YX. Figure 8 shows the MC placement we used in our 16-core and 64-core platforms, based on recommendations from [10] and [11] on MC placements.…”

“…The architecture discussed in [10] is called Direct all-to-all (DA2), while that from [11] is called XY-YX. Figure 8 shows the MC placement we used in our 16-core and 64-core platforms, based on recommendations from [10] and [11] on MC placements. We evaluated network latency, total application execution time and total energy consumption for all three architectures.…”

“…DA2 also has limitations due to serialization overhead which effects the scalability of their model in many cases. Jang et al [11] propose an MC placement approach and a virtual channel partitioning scheme in GPGPU NoCs. In their work, MCs are placed along the x axis of the mesh and flits use XY routing for requests and YX routing for replies, to reduce congestion at MCs and competition for links along the Y-axis.…”

“…Although NoC design has been studied in detail for more than a decade for CMPs [4][5][6][7], few efforts have analyzed and proposed viable NoC solutions for GPGPUs. The traffic pattern in GPGPUs is primarily many-to-few and few-to-many, with high volumes of traffic skewed towards memory replies [10], [11], [15]. Traditional meshbased NoC topologies used in CMPs are not capable of handling such skewed traffic effectively, leading to underutilized resources, which increases NoC latency, and congestion at MCs.…”