Unlocking wordline-level parallelism for fast inference on RRAM-based DNN accelerator

Park, Yeonhong; Lee, Seung Yul; Shin, Hoon; Heo, Jun; Ham, Tae Jun; Lee, Jae Woo

doi:10.1145/3400302.3415664

Cited by 6 publications

(1 citation statement)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Oobleck matches or significantly outperforms them for a wide range of model sizes and failure frequencies. [11,15,22,29,30,33,37,38,42,45,47,56]. However, they do not provide fault tolerance out-of-thebox and are orthogonal to Oobleck.…”

Section: Related Workmentioning

confidence: 99%

Oobleck: Resilient Distributed Training of Large Models Using Pipeline Templates

Jang,

Yang,

Zhang

et al. 2023

Proceedings of the 29th Symposium on Operating Systems Principles

View full text Add to dashboard Cite

Oobleck enables resilient distributed training of large DNN models with guaranteed fault tolerance. It takes a planningexecution co-design approach, where it first generates a set of heterogeneous pipeline templates and instantiates at least 𝑓 + 1 logically equivalent pipeline replicas to tolerate any 𝑓 simultaneous failures. During execution, it relies on alreadyreplicated model states across the replicas to provide fast recovery. Oobleck provably guarantees that some combination of the initially created pipeline templates can be used to cover all available resources after 𝑓 or fewer simultaneous failures, thereby avoiding resource idling at all times. Evaluation on large DNN models with billions of parameters shows that Oobleck provides consistently high throughput, and it outperforms state-of-the-art fault tolerance solutions like Bamboo and Varuna by up to 13.9×. CCS Concepts:• Computer systems organization → Dependable and fault-tolerant systems and networks; Distributed architectures; Neural networks.

show abstract