An analysis of on-chip interconnection networks for large-scale chip multiprocessors

Abstract

With the number of cores of chip multiprocessors (CMPs) rapidly growing as technology scales down, connecting the different components of a CMP in a scalable and efficient way becomes increasingly challenging. In this article, we explore the architectural-level implications of interconnection network design for CMPs with up to 128 fine-grain multithreaded cores. We evaluate and compare different network topologies using accurate simulation of the full chip, including the memory hierarchy and interconnect, and using a diverse set of scientific and engineering workloads.We find that the interconnect has a large impact on performance, as it is responsible for 60% to 75% of the miss latency. Latency, and not bandwidth, is the primary performance constraint, since, even with many threads per core and workloads with high miss rates, networks with enough bandwidth can be efficiently implemented for the system scales we consider. From the topologies we study, the flattened butterfly consistently outperforms the mesh and fat tree on all workloads, leading to performance advantages of up to 22%. We also show that considering interconnect and memory hierarchy together when designing large-scale CMPs is crucial, and neglecting either of the two can lead to incorrect conclusions. Finally, the effect of the interconnect on overall performance becomes more important as the number of cores increases, making interconnection choices especially critical when scaling up.

Holistic Routing Algorithm Design to Support Workload Consolidation in NoCs

“…The workloads can be classified into two groups: networkinsensitive applications and network-sensitive applications. Sophisticated routing algorithms can improve the network saturation throughput, but the full-system performance improvements depend on the load and traffic pattern created by each application [48]. Applications with a high network load and significant bursty traffic receive benefit from advanced routing algorithms.…”

Adaptive Backpressure: Efficient buffer management for on-chip networks

“…Performance in many CMPs workloads is primarily limited by latency rather than throughput [2]. In evaluating traffic isolation, we thus focus our investigation on how packet latency for a given foreground traffic is affected by a secondary background traffic.…”

“…As improvements to single-threaded performance have become limited by power constraints, this has caused industry focus to shift towards design approaches that scale performance by leveraging chiplevel parallelism. With future microprocessors expected to integrate hundreds of execution cores on a single die, on-chip communication will have a significant impact on chip-level performance and power efficiency [1], [2]. Networks-on-Chip (NoCs) are widely considered to be a promising approach for addressing the scalability and complexity challenges inherent in such designs [3].…”

Packet Chaining: Efficient Single-Cycle Allocation for On-Chip Networks

“…Previous work has observed that many applications in certain CMP configurations make light use of the network and thus are not affected by techniques improving throughput, like packet chaining [23]. In these cases, networks can reduce their cost, for example by narrowing their datapath, such that their average load increases and thus they become throughput-limited.…”

“…NoCs can consume considerable amounts of area and power, as well as affect application execution time [23]. Therefore, much research has focused on improving NoC efficiency.…”