A Hardware Accelerator for Protocol Buffers

Karandikar, Sagar; Leary, Chris; Kennelly, Chris; Zhao, Jerry; Parimi, Dinesh; Nikolić, Borivoje; Asanović, Krste; Ranganathan, Parthasarathy

doi:10.1145/3466752.3480051

Cited by 26 publications

(9 citation statements)

References 29 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We now describe three scenarios that depict how lack of visibility into accelerators' expected performance, makes designing and building systems that use them a challenging task. We use the term "accelerators" to refer to fixed-function ASICs whose functionality is baked into silicon, such as the TPU [34], the accelerators available on SoC-based Smart-NICs [7], Protoacc [36], and Inferentia [4]. We do not consider reprogrammable hardware such as FPGAs and GPUs.…”

Section: System Developers Are Flying Blind!mentioning

confidence: 99%

“…Currently, your stack runs on commodity servers, but you are considering offloading it to an accelerator. Your candidate hardware platforms include new servers with accelerators for RPC serialization/deserialization, such as Protoacc [36] or Optimus Prime [49] or one of several SmartNICs.…”

Section: System Developers Are Flying Blind!mentioning

confidence: 99%

“…With the decline of Moore's law, system designers are increasingly reliant on hardware accelerators for performance improvements. From datacenters to hand-held devices, hardware accelerators are used to speed up a wide variety of applications such as machine learning [4,33,34,45], video processing [21,54], compression, encryption [14,29] and system infrastructure tasks [5,22,26,36].…”

Section: Introductionmentioning

confidence: 99%

“…However, designing and building systems that use accelerators correctly (i.e., fully extract their performance benefits) is a challenging task, because developers have little to no visibility into an accelerator's expected performance behavior. Every accelerator bakes certain design choices into silicon, such as optimizing throughput over latency [49] or making assumptions about the typical workload [36], and if the software is a poor fit for these choices, running it on the accelerator may result in worse performance [37,40,41]. Since building an accelerator requires large teams and takes years, such efforts that miss their mark cost several million dollars [2].…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

The Case for Performance Interfaces for Hardware Accelerators

Iyer

Argyraki

et al. 2023

Proceedings of the 19th Workshop on Hot Topics in Operating Systems

View full text Add to dashboard Cite

While systems designers are increasingly turning to hardware accelerators for performance gains, realizing these gains is painstaking and error-prone. It can take several person-months to determine if a given accelerator is a good fit for a given piece of code, and accelerators that cost millions of dollars to build can slow down the very systems they were designed to accelerate.We argue that hardware accelerators must come with performance interfaces-interfaces that provide usable information about the accelerator's performance behavior just like semantic interfaces do for functionality-to facilitate their correct use. Since accelerators do not provide new functionality and are only useful if they improve system performance, performance interfaces are as integral to their correct use as semantic interfaces.

show abstract

Section: System Developers Are Flying Blind!mentioning

confidence: 99%

Section: System Developers Are Flying Blind!mentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

The Case for Performance Interfaces for Hardware Accelerators

Iyer

Argyraki

et al. 2023

Proceedings of the 19th Workshop on Hot Topics in Operating Systems

View full text Add to dashboard Cite

show abstract

“…To alleviate the interconnect (PCIe) overhead, a line of research integrates the entire NIC or accelerator to/near the CPU package [3,32,33,55,61,80,100,119,139]. Enjoying the benefit of fast NIC-core interaction, this approach has (1) high cost of designing and manufacturing them and (2) low flexibility of usage and maintaining.…”

Section: Related Workmentioning

confidence: 99%

ORCA: A Network and Architecture Co-design for Offloading us-scale Datacenter Applications

Yang¹,

Huang²,

Sun³

et al. 2022

Preprint

View full text Add to dashboard Cite

Responding to the "datacenter tax" and "killer microseconds" problems for datacenter applications, diverse solutions including Smart NIC-based ones have been proposed. Nonetheless, they often suffer from high overhead of communications over network and/or PCIe links. To tackle the limitations of the current solutions, this paper proposes ORCA, a holistic network and architecture co-design solution that leverages current RDMA and emerging cache-coherent off-chip interconnect technologies. Specifically, ORCA consists of four hardware and software components: (1) unified abstraction of inter-and intra-machine communications managed by one-sided RDMA write and cache-coherent memory write; (2) efficient notification of requests to accelerators assisted by cache coherence; (3) cache-coherent accelerator architecture directly processing requests received by NIC; and (4) adaptive device-tohost data transfer for modern server memory systems consisting of both DRAM and NVM exploiting state-of-the-art features in CPUs and PCIe. We prototype ORCA with a commercial system and evaluate three popular datacenter applications: in-memory key-value store, chain replication-based distributed transaction system, and deep learning recommendation model inference. The evaluation shows that ORCA provides 30.1∼69.1% lower latency, up to 2.5× higher throughput, and ∼ 3× higher power efficiency than the current state-of-the-art solutions.

show abstract