The benefits of general-purpose on-NIC memory

Pismenny, Boris; Liss, Liran; Morrison, Adam; Tsafrir, Dan

doi:10.1145/3503222.3507711

Cited by 18 publications

(3 citation statements)

References 51 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…DDIO has been proven to be effective [4,19,44,45,74,88,107,135,146,157,166,175], improving the performance of DRAM-based systems [4,19,88]. However, it does not always improve performance of NVM-based systems [74,166].…”

Section: Optimizing Device-host Data Transfer: Adaptive Ddiomentioning

confidence: 99%

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Yang¹,

Huang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Responding to the "datacenter tax" and "killer microseconds" problems for datacenter applications, diverse solutions including Smart NIC-based ones have been proposed. Nonetheless, they often suffer from high overhead of communications over network and/or PCIe links. To tackle the limitations of the current solutions, this paper proposes ORCA, a holistic network and architecture co-design solution that leverages current RDMA and emerging cache-coherent off-chip interconnect technologies. Specifically, ORCA consists of four hardware and software components: (1) unified abstraction of inter-and intra-machine communications managed by one-sided RDMA write and cache-coherent memory write; (2) efficient notification of requests to accelerators assisted by cache coherence; (3) cache-coherent accelerator architecture directly processing requests received by NIC; and (4) adaptive device-tohost data transfer for modern server memory systems consisting of both DRAM and NVM exploiting state-of-the-art features in CPUs and PCIe. We prototype ORCA with a commercial system and evaluate three popular datacenter applications: in-memory key-value store, chain replication-based distributed transaction system, and deep learning recommendation model inference. The evaluation shows that ORCA provides 30.1∼69.1% lower latency, up to 2.5× higher throughput, and ∼ 3× higher power efficiency than the current state-of-the-art solutions.

show abstract

Section: Optimizing Device-host Data Transfer: Adaptive Ddiomentioning

confidence: 99%

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Yang¹,

Huang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…Emerging SmartNIC frameworks propose a split-KVS design, where a small amount of NIC memory stores the KVS' hottest items [26,81], embodying the "small cache, big effect" principle [28]. Like NetCache [44], such designs focus on reads and are inefficient for write-intensive workloads, because writes are handled on a slow path to keep the CPU's main memory up-to-date.…”

Section: Sensitivity Analysismentioning

confidence: 99%

Cooperative Concurrency Control for Write-Intensive Key-Value Workloads

Sutherland

Falsafi

Daglis

2022

Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems,

View full text Add to dashboard Cite

Key-Value Stores (KVS) are foundational infrastructure components for online services. Due to their latency-critical nature, today's best-performing KVS contain a plethora of full-stack optimizations commonly targeting read-mostly, popularity-skewed workloads. Motivated by production studies showing the increased prevalence of write-intensive workloads, we break down the KVS workload space into four distinct classes, and argue that current designs are only sufficient for two of them. The reason is that KVS concurrency control protocols expose a fundamental tradeoff: avoiding synchronization by partitioning writes across threads is mandatory for high throughput, but necessarily creates load imbalance that grows with core count and write fraction. We break this tradeoff with C-4, a codesign between NIC hardware and KVS software that judiciously separates write requests into two classes: independent ones that can be balanced across threads, and dependent ones which must be queued. C-4 dynamically partitions independent writes with the NIC to increase the load balancing flexibility of current KVS designs, and adds a software layer to the KVS to compact dependent writes into batches. Our evaluation shows that for write-intensive workloads, C-4 reduces 99th% tail latency by 1.3 − 5× and improves throughput by up to 1.7×. CCS CONCEPTS• Hardware → Networking hardware; • Computer systems organization → Multicore architectures; Client-server architectures; • Networks → Network servers.

show abstract

“…To the best of our knowledge, Sherman is the first RDMA-based system that leverages on-chip memory of commodity RDMA NICs. A concurrent work leverages NIC's on-chip memory to accelerate network functions [50].…”

Section: Hierarchical On-chip Lockmentioning

confidence: 99%

Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory

Wang¹,

Lu²,

Shu³

2021

Preprint

View full text Add to dashboard Cite

Memory disaggregation architecture physically separates CPU and memory into independent components, which are connected via high-speed RDMA networks, greatly improving resource utilization of databases. However, such an architecture poses unique challenges to data indexing in databases due to limited RDMA semantics and near-zero computation power at memory-side. Existing indexes supporting disaggregated memory either suffer from low write performance, or require hardware modification.This paper presents Sherman, a write-optimized distributed B + Tree index on disaggregated memory that delivers high performance with commodity RDMA NICs. Sherman combines RDMA hardware features and RDMA-friendly software techniques to boost index write performance from three angles. First, to reduce round trips, Sherman coalesces dependent RDMA commands by leveraging in-order delivery property of RDMA. Second, to accelerate concurrent accesses, Sherman introduces a hierarchical lock that exploits on-chip memory of RDMA NICs. Finally, to mitigate write amplification, Sherman tailors the data structure layout of B + Tree with a two-level version mechanism. Our evaluation shows that, Sherman is one order of magnitude faster in terms of both throughput and 99th percentile latency on typical write-intensive workloads, compared with state-of-the-art designs.

show abstract

The benefits of general-purpose on-NIC memory

Cited by 18 publications

References 51 publications

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Cooperative Concurrency Control for Write-Intensive Key-Value Workloads

Sherman: A Write-Optimized Distributed B+Tree Index on Disaggregated Memory

Contact Info

Product

Resources

About