MemPool: A Shared-L1 Memory Many-Core Cluster with a Low-Latency Interconnect

Abstract

A key challenge in scaling shared-L1 multi-core clusters towards many-core (more than 16 cores) configurations is to ensure low-latency and efficient access to the L1 memory. In this work we demonstrate that it is possible to scale up the shared-L1 architecture: We present MemPool, a 32 bit manycore system with 256 fast RV32IMA "Snitch" cores featuring application-tunable execution units, running at 700 MHz in typical conditions (TT/0.80 V/25 °C). MemPool is easy to program, with all the cores sharing a global view of a large L1 scratchpad memory pool, accessible within at most 5 cycles. In MemPool's physical-aware design, we emphasized the exploration, design, and optimization of the low-latency processor-to-L1-memory interconnect. We compare three candidate topologies, analyzing them in terms of latency, throughput, and back-end feasibility. The chosen topology keeps the average latency at fewer than 6 cycles, even for a heavy injected load of 0.33 request/core/cycle. We also propose a lightweight addressing scheme that maps each core private data to a memory bank accessible within one cycle, which leads to performance gains of up to 20 % in real-world signal processing benchmarks. The addressing scheme is also highly efficient in terms of energy consumption since requests to local banks consume only half of the energy required to access remote banks. Our design achieves competitive performance with respect to an ideal, non-implementable full-crossbar baseline.

Spatz: A Compact Vector Processing Unit for High-Performance and Energy-Efficient Shared-L1 Clusters

“…Our vector unit is designed as the PE of a shared-L1 cluster, with a handful of MACUs per Spatz. We can further scale our design by replicating the shared-L1 cluster, connecting them with a lowlatency L1 interconnect [5]. This replication has two benefits.…”

“…MemPool [5] is a highly-parametric design. Its smallest unit, the tile (Figure 1), contains four Snitch cores, 2 KiB of L1 I$, 16 KiB of SPM divided into 16 SRAM banks, and a fully-connected logarithmic crossbar between the cores and memories.…”

“…Its smallest unit, the tile (Figure 1), contains four Snitch cores, 2 KiB of L1 I$, 16 KiB of SPM divided into 16 SRAM banks, and a fully-connected logarithmic crossbar between the cores and memories. This tile can be replicated to build systems with as low as 16 cores (the "minpool" configuration) to as high as 256 cores (the "mempool" configuration) [5,16]. We analyze Spatz' impact on the PPA of several MemPool configurations, representing a wide range of shared-L1 cluster sizes.…”

“…A common architectural pattern comprises clusters of Processing Elements (PEs) that share tightly-coupled L1 memory through a low-latency interconnect [5]. Graphics Processing Units (GPUs), which dominate most of the Top500 list [21], follow this architectural pattern.…”

“…Each chiplet contains 128 clusters, each featuring 128 KiB of tightly-coupled L1 memory and eight small Snitch cores equipped with large double-precision Floating Point Units (FPUs). Another example is MemPool [5], a scaled-up Snitchbased Snitch with 256 cores sharing 1 MiB of L1 Scratchpad Memory (SPM) accessible by all cores within at most five cycles of zero-load latency. However, there is a fundamental trade-off between the number of PEs and the Von Neumann Bottleneck (VNB), i.e., the memory traffic and energy overhead due to the instruction fetching mechanism and related logic [4].…”