Virtual-Link: A Scalable Multi-Producer, Multi-Consumer Message Queue Architecture for Cross-Core Communication

Abstract

Cross-core communication is becoming a bottleneck as the number of processing elements (PEs) increase per systemon-chip (SoC). Typical hardware solutions are often inflexible; while software solutions are flexible, they have performance scaling limitations. A key problem to solve, as we will show, is that of shared state in any software-based message queue mechanism. In this paper, we propose Virtual-Link (VL), a novel light-weight communication mechanism with hardware support to facilitate data movement between multiple data producers and consumers. VL reduces the amount of coherent shared state for synchronization, which is a key bottleneck for many approaches, to zero. VL provides further performance benefit by keeping data on the fast path (i.e., within the on-chip interconnect). VL enables directed cache-injection (stashing) between PEs on the coherence bus, reducing the latency for core-to-core communication. VL is particularly effective for fine-grain tasks on streaming data. Evaluation on a full system simulator with 6 benchmarks shows that VL achieves 2.65× speedup over state-of-the-art softwarebased communication mechanisms, and reduces memory traffic down to 34%.