NoCMsg: Scalable NoC-Based Message Passing

Abstract

Abstract-Current processor design with ever more cores may ensure that theoretical compute performance still follows past increases (resting from Moore's law), but they also increasingly present a challenge to hardware and software alike. As the core count increases, the network-on-chip (NoC) topology has changed from buses over rings and fully connected meshes to 2D meshes. The question is which programming paradigm provides the scalability needed to ensure performance is close to theoretical peak, where 2D meshes provide the most scalable design to date.This work contributes NoCMsg, a low-level message passing abstraction over NoCs. NoCMsg is specifically designed for large core counts in 2D meshes. Its design ensures deadlock free messaging for wormhole Manhattan-path routing over the NoC. Experimental results on the TilePro hardware platform show that NoCMsg can significantly reduce communication times by up to 86% for single packet messages and up to 40% for larger messages compared to other NoC-based message approaches. Results further demonstrate the potential of NoC messaging to outperform shared memory abstractions by up to 93% as core counts and inter-process communication increase, i.e., we observe that shared memory scales up to about 16 cores while message passing performs well beyond that threshold on this platform. To the best of our knowledge, this is the first head-on comparison of shared memory and advanced message passing specifically designed for NoCs on an actual hardware platform with larger core counts on a single socket.

NoCMsg

“…Our study is based on native shared memory at the DRAM level that does not require load/stores to be rewritten, nor is it restricted to 8KB shared data. We recently ported NoCMsg to the Intel SCC [Patil 2014] and found that our flow control elimination in conjunction with contention-free communication patterns provide similar performance improvements on the Intel SCC compared to the Tilera [Zimmer and Mueller 2014;Yagna 2013].…”

Distributed Job Allocation for Large-Scale Manycores

“…The User Dynamic Network (UDN) interconnect is the only one available for user-generated messages. We use the services of the NoCMsg [33] library. NoCMsg provides a deadlock free, scalable and efficient low-level message passing layer over UDN with an MPI like interface.…”

“…With SSI, resources are aggregated to present a single view of the operating system environment while data access and communication are realized via shared memory over traditional bidirectional buses. This approach delivers some performance increases in the natural evolution from single core up to 16 cores, but it deteriorates rapidly when the number of cores increases further [33].…”

“…Cache misses and coherence updates are also extremely expensive in SSI approaches on multicores since each core has its own cache that must be coherent with shared memory, as well as with other cores. Recent work by Baumann et al [6], Wentzlaff et al [29] and Zimmer et al [33] show that coherent shared memory may not scale well to large core counts. They instead promote the usage of scalable message passing for OS communication in large-scale manycores.…”

“…We follow the core of the design principles postulated by Peter et al [24] for designing multi-core schedulers. We even go one level further and take a purely distributed message passing approach as the primary means of communication enabled by our adoption of NoCMsg [33] as our low-level messaging library.…”