An exact algorithm for sparse matrix bipartitioning

Abstract

The sparse matrix partitioning problem arises when minimizing communication in parallel sparse matrix-vector multiplications. Since the problem is NP-hard, heuristics are usually employed to find solutions. Here, we present a purely combinatorial branch-and-bound method for computing optimal bipartitionings of sparse matrices, in the sense that they have the lowest communication volume out of all possible bipartitionings obeying a certain load balance constraint. The method is based on a way of partitioning similar to the recently proposed medium-grain heuristic, which reduces the number of solutions to be considered in the branch-and-bound method.We applied the proposed optimal bipartitioner to find the optimal communication volume of all matrices of the University of Florida sparse matrix collection with 1000 nonzeros or less. For 85% of the matrices, an optimal bipartitioning was found within a single day of computation and for 58% even within a second. We also present optimal results for selected larger matrices, up to 129,042 nonzeros. The optimal bipartitionings and corresponding communication volumes are made publicly available in a benchmark collection.

Optimally bipartitioning sparse matrices with reordering and parallelization

“…In addition, these have not proven capable of tackling the optimal sparse‐matrix bipartitioning problem for matrices with reasonable sizes. A short review of these optimal graph‐ and hypergraph‐partitioning literature can be found in the work of Pelt and Bisseling …”

“…The tool MondriaanOpt is developed by Pelt and Bisseling; it gets a sparse matrix and an upper bound for the imbalance ratio ε as inputs. It outputs the optimal partitioning.…”

“…We used each ordering type in two different modes. Given a node in the tree (ie, a row/column to be assigned), MondriaanOpt explores three options for that node; part 1, part 2, or cut (we refer the reader to Pelt and Bisseling for more details). We modified MondriaanOpt to evaluate the impact of this option for our ordering heuristics.…”

“…Recently, a purely combinatorial branch‐and‐bound–based optimal matrix bipartitioning approach has been proposed by Pelt and Bisseling . This approach reduces the complexity by exploring the rows/columns instead of the nonzeros of the sparse matrix since the number of nonzeros is much more than the number of rows and columns . Following this idea, they have implemented a publicly available tool, MondriaanOpt.…”

Exact k-way sparse matrix partitioning

“…A recent exact solver for the sparse matrix partitioning problem is MondriaanOpt [12], a branch-and-bound (BB) bipartitioner that branches on the possible choices for a row or column, which is either completely assigned to processor 0, or completely assigned to processor 1, or cut, meaning that some nonzeros are assigned to processor 0 and others to processor 1, where the choice of nonzeros need not be specified. MondriaanOpt prunes the solution tree by lower bounds on the communication volume of a partial solution, based on cuts that are either explicitly or implicitly present.…”