A Tensor Product Formulation of Strassen′s Matrix Multiplication Algorithm with Memory Reduction

Abstract

In this article, we present a program generation strategy of Strassen's matrix multiplication algorithm using a programming methodology based on tensor product formulas. In this methodology, block recursive programs such as the fast Fourier Transforms and Strassen's matrix multiplication algorithm are expressed as algebraic formulas involving tensor products and other matrix operations. Such formulas can be systematically translated to high-performance parallel/vector codes for various architectures. In this article, we present a nonrecursive implementation of Strassen's algorithm for shared memory vector processors such as the Cray Y-MP. A previous implementation of Strassen's algorithm synthesized from tensor product formulas required working storage of size 0(7n) for multiplying 2n x 2n matrices. We present a modified formulation in which the working storage requirement is reduced to 0(4n). The modified formulation exhibits sufficient parallelism for efficient implementation on a shared memory multiprocessor. Performance results on a Cray Y-MPB/64 are presented.

A framework for practical parallel fast matrix multiplication

“…Strassen's fast matrix multiplication algorithm has been implemented for both shared-memory [8,21] and distributed-memory architectures [2,11,24]. For our parallel algorithms in Section 4, we use the ideas of breadth-first and depth-first traversals of the recursion trees, which were first considered by Kumar et al [21] and Ballard et al [2] for minimizing memory footprint and communication.…”

Communication-optimal parallel algorithm for strassen's matrix multiplication

“…Kumar, Huang, Johnson, and Sadayappan [24] implemented Strassen's algorithm on a shared-memory machine. They identified the tradeoff between available parallelism and total memory footprint by differentiating between "partial" and "complete" evaluation of the algorithm, which corresponds to what we call depth-first and breadth-first traversal of the recursion tree (see Section 3.1).…”

Stark: Fast and Scalable Strassen’s Matrix Multiplication Using Apache Spark

“…An extensive literature exists on parallelizing naive matrix multiplication algorithms [9], [10], [11], [12], [13], [6], [14], [15], [16], [17], [5], [18], [19], [20] and [21]. Similarly Strassen's matrix multiplication algorithm has also been extensively studied for parallelization [22], [23], [24], [25], [16], [26], [27], [28], [29], [30] and [31]. As pointed out in [6], [20] and [30], the literature on parallel and distributed matrix multiplication can be divided broadly into three categories: 1) Grid based approach, 2) BFS/DFS based approach and 3) Hadoop and Spark based approach.…”