Extreme Datacenter Specialization for Planet-Scale Computing

Xie, Shijie; Davidson, Scott; Magaki, Ikuo; Khazraee, Moein; Vega, Luis; Zhang, Lu; Taylor, Michael

doi:10.1145/3273982.3273991

Cited by 14 publications

(6 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…First, as D. Mackenzie (2021) has noticed, every material political economy, by virtue of its materiality, produces specific ‘ spatial materialities’ (p. 12 original emphasis). In both blockchain and AI, this spatial materiality instantiated itself in clouds understood as large-scale assemblages for ‘planet-scale computing’ ( Xie et al, 2018 ), with Bitcoin ASICs and Google TPUs being the cutting edge of those industries. However, cloud computing is more than just an assemblage of datacentres ( Amoore, 2018 ), but is instead a highly flexible arrangement ( Narayan, 2022 ).…”

Section: Conclusion: Going Full Stackmentioning

confidence: 99%

Close to the metal: Towards a material political economy of the epistemology of computation

Rella

2023

Soc Stud Sci

View full text Add to dashboard Cite

This paper investigates the role of the materiality of computation in two domains: blockchain technologies and artificial intelligence (AI). Although historically designed as parallel computing accelerators for image rendering and videogames, graphics processing units (GPUs) have been instrumental in the explosion of both cryptoasset mining and machine learning models. The political economy associated with video games and Bitcoin and Ethereum mining provided a staggering growth in performance and energy efficiency and this, in turn, fostered a change in the epistemological understanding of AI: from rules-based or symbolic AI towards the matrix multiplications underpinning connectionism, machine learning and neural nets. Combining a material political economy of markets with a material epistemology of science, the article shows that there is no clear-cut division between software and hardware, between instructions and tools, and between frameworks of thought and the material and economic conditions of possibility of thought itself. As the microchip shortage and the growing geopolitical relevance of the hardware and semiconductor supply chain come to the fore, the paper invites social scientists to engage more closely with the materialities and hardware architectures of ‘virtual’ algorithms and software.

show abstract

Section: Conclusion: Going Full Stackmentioning

confidence: 99%

Close to the metal: Towards a material political economy of the epistemology of computation

Rella

2023

Soc Stud Sci

View full text Add to dashboard Cite

show abstract

“…Recent work such as ASIC Clouds [62,99] has used design space exploration to optimize directly for datacenter total cost of ownership in the context of bitcoin mining, video transcoding, and machine learning accelerators. FAST extends this by considering return-oninvestment (ROI) and using ROI to demonstrate production feasibility of FAST-generated designs.…”

Section: Related Workmentioning

confidence: 99%

“…Our example ROI calculation assumes a NVIDIA DGX A100 320GB platform baseline containing 8x A100 accelerators with a manufacturer's suggested price (MSRP) of $199,000 [87]. We assume the May 2021 average price of electricity for the US Commercial sector ($0.1084/kWh) from the US Energy Information Administration [1], an accelerator deployment lifetime of 3 years [89], the cost per engineer based on the reported median total compensation for a SWE working in the San Francisco bay area ($240,000) [3] with a 65% salary overhead [99], and all other values from previous work [99]. Since our experimental results assume a sub-10nm process technology, we extrapolate wafer mask and PHY IP costs using exponential scaling as observed in [99].…”

Section: The Economics Of Specialized Acceleratorsmentioning

confidence: 99%

A full-stack search technique for domain optimized deep learning accelerators

Zhang

Huda

Songhori

et al. 2022

Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

View full text Add to dashboard Cite

The rapidly-changing deep learning landscape presents a unique opportunity for building inference accelerators optimized for specific datacenter-scale workloads. We propose Full-stack Accelerator Search Technique (FAST), a hardware accelerator search framework that defines a broad optimization environment covering key design decisions within the hardware-software stack, including hardware datapath, software scheduling, and compiler passes such as operation fusion and tensor padding. In this paper, we analyze bottlenecks in stateof-the-art vision and natural language processing (NLP) models, including EfficientNet [91] and BERT [19], and use FAST to design accelerators capable of addressing these bottlenecks. FAST-generated accelerators optimized for single workloads improve Perf/TDP by 3.7× on average across all benchmarks compared to TPU-v3. A FASTgenerated accelerator optimized for serving a suite of workloads improves Perf/TDP by 2.4× on average compared to TPU-v3. Our return on investment analysis shows that FAST-generated accelerators can potentially be practical for moderate-sized datacenter deployments. CCS CONCEPTS• Hardware → Electronic design automation; • Computer systems organization → Parallel architectures.

show abstract

“…Lack of floating-point: PDP hardware, being designed solely for efficient packet processing, lacks floating-point arithmetic support even in more general purpose NPU SmartNICs. Luckily, the embedded ML literature offers many low-precision floatingpoint formats [24,42,49,51] and fixed-point representations [35,54]. The latter set requires only integer arithmetic, which allows us to express and update policy parameters in-NIC.…”

Section: Design and Implementationmentioning

confidence: 99%

Revisiting the Classics: Online RL in the Programmable Dataplane

Simpson

Pezaros

2022

NOMS 2022-2022 IEEE/IFIP Network Operations and Management Symposium

View full text Add to dashboard Cite

Data-driven networking is becoming more capable and widely researched, partly driven by the efficacy of Deep Reinforcement Learning (DRL) algorithms. Yet the complexity of both DRL inference and learning force these tasks to be pushed away from the dataplane to hosts, harming latency-sensitive applications. Online learning of such policies cannot occur in the dataplane, despite being useful techniques when problems evolve or are hard to model.We present OPaL-On Path Learning-the first work to bring online reinforcement learning to the dataplane. OPaL makes online learning possible in constrained SmartNIC hardware by returning to classical RL techniques-avoiding neural networks. Our design allows weak yet highly parallel SmartNIC NPUs to be competitive against commodity x86 hosts, despite having fewer features and slower cores. Compared to hosts, we achieve a 21 × reduction in 99.99 th tail inference times to 34 µs, and 9.9 × improvement in online throughput for real-world policy designs. In-NIC execution eliminates PCIe transfers, and our asynchronous compute model ensures minimal impact on traffic carried by a co-hosted P4 dataplane. OPaL's design scales with additional resources at compile-time to improve upon both decision latency and throughput, and is quickly reconfigurable at runtime compared to reinstalling device firmware.

show abstract

Extreme Datacenter Specialization for Planet-Scale Computing

Cited by 14 publications

References 28 publications

Close to the metal: Towards a material political economy of the epistemology of computation

Close to the metal: Towards a material political economy of the epistemology of computation

A full-stack search technique for domain optimized deep learning accelerators

Revisiting the Classics: Online RL in the Programmable Dataplane

Contact Info

Product

Resources

About