SpTFS: Sparse Tensor Format Selection for MTTKRP via Deep Learning

Sun, Qingxiao; Liu, Yi; Dun, Ming; Yang, Hailong; Luan, Zhongzhi; Gan, Lin; Yang, Guangwen; Qian, Depei

doi:10.1109/sc41405.2020.00022

Cited by 10 publications

(3 citation statements)

References 50 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…k2 (1) kK( 1 format selection of sparse tensor kernels. SpTFS [26] explores deep learning to select a sparse tensor format. No prior work has studied machine learning techniques for general SpMM tuning.…”

Section: Input-adaptive Kernel Selectionmentioning

confidence: 99%

Heuristic Adaptability to Input Dynamics for SpMM on GPUs

Dai¹,

Huang²,

Shang³

et al. 2022

Preprint

View full text Add to dashboard Cite

Sparse Matrix-Matrix Multiplication (SpMM) has served as fundamental components in various domains. Many previous studies exploit GPUs for SpMM acceleration because GPUs provide high bandwidth and parallelism. We point out that a static design does not always improve the performance of SpMM on different input data (e.g., >85% performance loss with a single algorithm). In this paper, we consider the challenge of input dynamics from a novel autotuning perspective, while following issues remain to be solved: (1) Orthogonal design principles considering sparsity. Orthogonal design principles for such a sparse problem should be extracted to form different algorithms, and further used for performance tuning.(2) Nontrivial implementations in the algorithm space. Combining orthogonal design principles to create new algorithms needs to tackle with new challenges like thread race handling. (3) Heuristic adaptability to input dynamics. The heuristic adaptability is required to dynamically optimize code for input dynamics.To tackle these challenges, we first propose a novel three-loop model to extract orthogonal design principles for SpMM on GPUs. The model not only covers previous SpMM designs, but also comes up with new designs absent from previous studies. We propose techniques like conditional reduction to implement algorithms missing in previous studies. We further propose DA-SpMM, a Data-Aware heuristic GPU kernel for SpMM. DA-SpMM adaptively optimizes code considering input dynamics. Extensive experimental results show that, DA-SpMM achieves 1.26×∼1.37× speedup compared with the best NVIDIA cuSPARSE algorithm on average, and brings up to 5.59× end-to-end speedup to applications like Graph Neural Networks. * Example performance (GFLOP/S) ** Normalized performance to the algorithm with the best performance * * * Geometric mean of all normalized performance for different matrices **** Upper bound if we use the best algorithm for each matrix posteriorly

show abstract

Section: Input-adaptive Kernel Selectionmentioning

confidence: 99%

Heuristic Adaptability to Input Dynamics for SpMM on GPUs

Dai¹,

Huang²,

Shang³

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…A wealth of work perform MTTKRP computations on distributed-memory platforms using MPI [11,22,43,46,48], or the MapReduce [7,21] framework. Other studies [49,50,54] explore format selection based on machine learning models to efficiently leverage existing sparse formats.…”

Section: Related Workmentioning

confidence: 99%

Efficient, Out-of-Memory Sparse MTTKRP on Massively Parallel Architectures

Nguyen,

Helal,

Checconi

et al. 2022

Preprint

View full text Add to dashboard Cite

Tensor decomposition (TD) is an important method for extracting latent information from high-dimensional (multi-modal) sparse data. This study presents a novel framework for accelerating fundamental TD operations on massively parallel GPU architectures. In contrast to prior work, the proposed Blocked Linearized CoOrdinate (BLCO) format enables efficient out-of-memory computation of tensor algorithms using a unified implementation that works on a single tensor copy. Our adaptive blocking and linearization strategies not only meet the resource constraints of GPU devices, but also accelerate data indexing, eliminate control-flow and memory-access irregularities, and reduce kernel launching overhead. To address the substantial synchronization cost on GPUs, we introduce an opportunistic conflict resolution algorithm that discovers and resolves conflicting updates across threads on-the-fly, without keeping any auxiliary information or storing non-zero elements in specific mode orientations. As a result, our framework delivers superior in-memory performance compared to prior state-of-the-art, and is the only framework capable of processing out-of-memory tensors. On the latest Intel and NVIDIA GPUs, BLCO achieves 2.12 − 2.6× geometric-mean speedup (with up to 33.35× speedup) over the state-of-the-art mixed-mode compressed sparse fiber (MM-CSF) on a range of real-world sparse tensors.

show abstract

“…A wealth of work perform MTTKRP computations on distributedmemory platforms using MPI [11,22,45,48,50], or the MapReduce [7,21] framework. Other studies [51,52,56] explore format selection based on machine learning models to efficiently leverage existing sparse formats.…”

Section: Related Workmentioning

confidence: 99%

Efficient, out-of-memory sparse MTTKRP on massively parallel architectures

Nguyen¹,

Helal

Checconi

et al. 2022

Proceedings of the 36th ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Tensor decomposition (TD) is an important method for extracting latent information from high-dimensional (multi-modal) sparse data. This study presents a novel framework for accelerating fundamental TD operations on massively parallel GPU architectures. In contrast to prior work, the proposed Blocked Linearized CoOrdinate (BLCO) format enables efficient out-of-memory computation of tensor algorithms using a unified implementation that works on a single tensor copy. Our adaptive blocking and linearization strategies not only meet the resource constraints of GPU devices, but also accelerate data indexing, eliminate control-flow and memoryaccess irregularities, and reduce kernel launching overhead. To address the substantial synchronization cost on GPUs, we introduce an opportunistic conflict resolution algorithm, in which threads collaborate instead of contending on memory access to discover and resolve their conflicting updates on-the-fly, without keeping any auxiliary information or storing non-zero elements in specific mode orientations. As a result, our framework delivers superior in-memory performance compared to prior state-of-the-art, and is the only framework capable of processing out-of-memory tensors. On the latest Intel and NVIDIA GPUs, BLCO achieves 2.12 − 2.6× geometric-mean speedup (with up to 33.35× speedup) over the state-of-the-art mixed-mode compressed sparse fiber (MM-CSF) on a range of real-world sparse tensors. CCS CONCEPTS• Mathematics of computing → Mathematical software performance; • Computing methodologies → Massively parallel algorithms.

show abstract

SpTFS: Sparse Tensor Format Selection for MTTKRP via Deep Learning

Cited by 10 publications

References 50 publications

Heuristic Adaptability to Input Dynamics for SpMM on GPUs

Heuristic Adaptability to Input Dynamics for SpMM on GPUs

Efficient, Out-of-Memory Sparse MTTKRP on Massively Parallel Architectures

Efficient, out-of-memory sparse MTTKRP on massively parallel architectures

Contact Info

Product

Resources

About