Flexible Group-Level Pruning of Deep Neural Networks for On-Device Machine Learning

Kwang-Bae, Lee; Kim, Hoseung; Lee, Hayun; Shin, Dongkun

doi:10.23919/date48585.2020.9116287

Cited by 10 publications

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The unaligned group-level pruning proposed in [24] increases the accuracy of this kind of semi-structured pruning techniques by providing additional flexibility. N VIDIA cuSparse [29] is a library from NVIDIA that implements several linear algebra routines for sparse matrices stored in different compressed formats (COO, CSR and Blocked-Ellpack).…”

Section: Related Workmentioning

confidence: 99%

VENOM: A Vectorized N:M Format for Unleashing the Power of Sparse Tensor Cores

Castro,

Ivanov,

Andrade

et al. 2023

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

View full text Add to dashboard Cite

The increasing success and scaling of Deep Learning models demands higher computational efficiency and power. Sparsification can lead to both smaller models as well as higher compute efficiency, and accelerated hardware is becoming available. However, exploiting it efficiently requires kernel implementations, pruning algorithms, and storage formats, to utilize hardware support of specialized sparse vector units. An example of those are the NVIDIA's Sparse Tensor Cores (SPTCs), which promise a 2× speedup. However, SPTCs only support the 2:4 format, limiting achievable sparsity ratios to 50%. We present the V:N:M format, which enables the execution of arbitrary N:M ratios on SPTCs. To efficiently exploit the resulting format, we propose Spatha, a high-performance sparselibrary for DL routines. We show that Spatha achieves up to 37× speedup over cuBLAS. We also demonstrate a second-order pruning technique that enables sparsification to high sparsity ratios with V:N:M and little to no loss in accuracy in modern transformers.

show abstract

Section: Related Workmentioning

confidence: 99%