A General Sparse Tensor Framework for Electronic Structure Theory

Manzer, Samuel F.; Epifanovsky, Evgeny; Krylov, Anna I.; Head‐Gordon, Martin

doi:10.1021/acs.jctc.6b00853

Cited by 13 publications

(10 citation statements)

References 41 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The conventional approaches first extract dense block-pairs of the two input tensors, and then perform multiplication by calling dense BLAS linear algebra. Finally, those approaches pre-allocate the output tensor using domain knowledge or a symbolic phase approach [20,25,53,54,67], such as TiledArray [54], Cyclops Tensor Framework [36], and libtensor [14,49]. The state-of-the-art work Sparta focuses on element-wise sparse tensor contractions [44], solving the high dimensionality challenges through hash tablebased approaches and addressing the unknown output tensor and irregular memory access challenges by dynamic allocation, permutation and sorting.…”

Section: Related Workmentioning

confidence: 99%

“…High-order sparse tensors have been studied well in tensor decomposition on various hardware platforms [7, 26, 37-40, 43, 51, 52, 55, 69-71] with a focus on the product of a sparse tensor and a dense matrix or vector. The two sparse tensor contraction (SpTC) has been studied well [14,20,25,36,44,49,53,54,54,67] where block-wise sparsity is the main focus. As the needs of element-/pair-wise sparsity emerge in applications from chemistry, physics and deep learning [4,17,31,41,62,63], the recent work [44] studied element-wise SpTC.…”

Section: Introductionmentioning

confidence: 99%

“…The memory capacity problem in an SpTCSeq becomes much more serious than in an individual SpTC. This memory capacity problem is especially pronounced in those HPC applications with increasing dimension sizes of tensors [4,9,14,17,49,54,72]. Expanding DRAM capacity is not cost-effective, while adding cheap but much slower Solid-State Drive (SSD) causes significant performance drop.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Athena

Liu

Gioiosa

et al. 2021

Proceedings of the ACM International Conference on Supercomputing

View full text Add to dashboard Cite

Sparse tensor contraction sequence has been widely employed in many fields, such as chemistry and physics. However, how to efficiently implement the sequence faces multiple challenges, such as redundant computations and memory operations, massive memory consumption, and inefficient utilization of hardware. To address the above challenges, we introduce Athena, a high-performance framework for SpTC sequences. Athena introduces new data structures, leverages emerging Optane-based heterogeneous memory (HM) architecture, and adopts stage parallelism. In particular, Athena introduces shared hash table-represented sparse accumulator to eliminate unnecessary input processing and data migration; Athena uses a novel data-semantic guided dynamic migration solution to make the best use of the Optane-based HM for high performance; Athena also co-runs execution phases with different characteristics to enable high hardware utilization. Evaluating with 12 datasets, we show that Athena brings 327-7362× speedup over the state-ofthe-art SpTC algorithm. With the dynamic data placement guided by data semantics, Athena brings performance improvement on Optane-based HM over a state-of-the-art software-based data management solution, a hardware-based data management solution, and PMM-only by 1.58×, 1.82×, and 2.34× respectively. Athena also showcases its effectiveness in quantum chemistry and physics scenarios. CCS CONCEPTS• Mathematics of computing → Mathematical software performance; • Computing methodologies → Shared memory algorithms.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Athena

Liu

Gioiosa

et al. 2021

Proceedings of the ACM International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…Our work on graph optimization builds on substantial efforts for optimization of computational graphs of tensor operations. Tensor contraction can be optimized via parallelization [22,23,41,49], efficient transposition [51], blocking [10,18,28,43], exploiting symmetry [15,48,49], and sparsity [22,24,32,39,39,47]. For complicated tensor graphs, specialized compilers like XLA [52] and TVM [8] rewrite the computational graph to optimize program execution and memory allocation on dedicated hardware.…”

Section: Previous Workmentioning

confidence: 99%

AutoHOOT

Solomonik

2020

Proceedings of the ACM International Conference on Parallel Architectures and Compilation Techniques

View full text Add to dashboard Cite

High-order optimization methods, including Newton's method and its variants as well as alternating minimization methods, dominate the optimization algorithms for tensor decompositions and tensor networks. These tensor methods are used for data analysis and simulation of quantum systems. In this work, we introduce Auto-HOOT, the first automatic differentiation (AD) framework targeting at high-order optimization for tensor computations. AutoHOOT takes input tensor computation expressions and generates optimized derivative expressions. In particular, AutoHOOT contains a new explicit Jacobian / Hessian expression generation kernel whose outputs maintain the input tensors' granularity and are easy to optimize. The expressions are then optimized by both the traditional compiler optimization techniques and specific tensor algebra transformations. Experimental results show that AutoHOOT achieves competitive CPU and GPU performance for both tensor decomposition and tensor network applications compared to existing AD software and other tensor computation libraries with manually written kernels. The tensor methods generated by AutoHOOT are also well-parallelizable, and we demonstrate good scalability on a distributed memory supercomputer.

show abstract

“…Lewis et al introduced a clustered low-rank tensor format to exploit element and rank sparsities [81]. Block sparsity has been utilized in coupled-cluster singles and doubles (CCSD) in the work [15,37,64,91,107]. Scaled opposite spin second order Mller-Plesset perturbation theory (SOS-MP2) method uses tensor hypercontraction (T ), approximating a electron Coulomb repulsion integrals (ERI) tensor by decomposing into lower order tensors, with sparsity [133].…”

Section: 24mentioning

confidence: 99%

PASTA: a parallel sparse tensor algorithm benchmark suite

et al. 2019

CCF Trans. HPC

View full text Add to dashboard Cite

Tensor methods have gained increasingly a ention from various applications, including machine learning, quantum chemistry, healthcare analytics, social network analysis, data mining, and signal processing, to name a few. Sparse tensors and their algorithms become critical to further improve the performance of these methods and enhance the interpretability of their output. is work presents a sparse tensor algorithm benchmark suite (PASTA) for single-and multi-core CPUs. To the best of our knowledge, this is the rst benchmark suite for sparse tensor world. PASTA targets on: 1) helping application users to evaluate di erent computer systems using its representative computational workloads; 2) providing insights to be er utilize existed computer architecture and systems and inspiration for the future design. is benchmark suite will be publicly released. ACM Reference format:

show abstract

A General Sparse Tensor Framework for Electronic Structure Theory

Cited by 13 publications

References 41 publications

Athena

Athena

AutoHOOT

PASTA: a parallel sparse tensor algorithm benchmark suite

Contact Info

Product

Resources

About