CINM (Cinnamon): A Compilation Infrastructure for Heterogeneous Compute In-Memory and Compute Near-Memory Paradigms

Abstract

The rise of data-intensive applications exposed the limitations of conventional processor-centric von-Neumann architectures that struggle to meet the off-chip memory bandwidth demand. Therefore, recent innovations in computer architecture advocate computein-memory (CIM) and compute-near-memory (CNM), non-von-Neumann paradigms achieving orders-of-magnitude improvements in performance and energy consumption. Despite significant technological breakthroughs in the last few years, the programmability of these systems is still a serious challenge. Their programming models are too low-level and specific to particular system implementations. Since such future architectures are predicted to be highly heterogenous, developing novel compiler abstractions and frameworks become necessary. To this end, we present CINM (Cinnamon), a first end-to-end compilation flow that leverages the hierarchal abstractions to generalize over different CIM and CNM devices and enable device-agnostic and device-aware optimizations. Cinnamon progressively lowers input programs and performs optimizations at each level in the lowering pipeline. To show its efficacy, we evaluate CINM on a set of benchmarks for the well-known UPMEM CNM system and the memristors-based CIM accelerators. We show that Cinnamon, supporting multiple hardware targets, generates highperformance code comparable to or better than state-of-the-art implementations. CCS CONCEPTS• Hardware → Emerging architectures; Emerging tools and methodologies; Emerging languages and compilers; • Computing methodologies → Parallel computing methodologies.1 PIM, CIM, and in-memory computing (IMC) are used alternatively in the literature. We will use CIM in this paper.

Programming Model Extensions for General-Purpose Processing-In-Memory

“…Additionally, Samsung [6] and SK Hynix [28] have introduced specialized PIM designed significantly for machine learning tasks. These systems utilize HBM2 and GDDR6 DRAM standards, offering support for high-performance operations in the range of TFLOPS [29]. Especially, in [6], the I/O boundary of a bank is equipped with a PIM execution unit comprising SIMD floating-point units, command register files (CRF), general register files (GRF), and scalar register files (SRF).…”

“…To enhance the expressiveness of new devices, CINM [29] proposed a general end-to-end compilation infrastructure for heterogeneous computing-in-memory (CIM) and compute-near-memory (CNM) devices. Similar to our design, it uses MLIR rewriting and introduces reusable abstractions and components.…”