Mapping Dense LU Factorization on Multicore Supercomputer Nodes

Abstract

Abstract-Dense LU factorization is a prominent benchmark used to rank the performance of supercomputers. Many implementations use block-cyclic distributions of matrix blocks onto a two-dimensional process grid. The process grid dimensions drive a trade-off between communication and computation and are architecture-and implementation-sensitive. The critical panel factorization steps can be made less communication-bound by overlapping asynchronous collectives for pivoting with the computation of rank-k updates. By shifting the computationcommunication trade-off, a modified block-cyclic distribution can beneficially exploit more available parallelism on the critical path, and reduce panel factorization's memory hierarchy contention on now-ubiquitous multicore architectures.During active panel factorization, rank-1 updates stream through memory with minimal reuse. In a column-major process grid, the performance of this access pattern degrades as too many streaming processors contend for access to memory. A blockcyclic mapping in the row-major order does not encounter this problem, but consequently sacrifices node and network locality in the critical pivoting steps. We introduce striding to vary between the two extremes of row-and column-major process grids.The maximum available parallelism in the critical path work (active panel factorization, triangular solves, and subsequent broadcasts) is bounded by the length or width of the process grid. Increasing one dimension of the process grid decreases the number of distinct processes and nodes in the other dimension. To increase the harnessed parallelism in both dimensions, we start with a tall process grid. We then apply periodic rotation to this grid to restore exploited parallelism along the row to previous levels.As a test-bed for further mapping experiments, we describe a dense LU implementation that allows a block distribution to be defined as a general function of block to processor. Other mappings can be tested with only small, local changes to the code.

Controlling Concurrency and Expressing Synchronization in Charm++ Programs

“…In dense LU factorization, the matrix being factorized is decomposed into a 2D grid of blocks, which in the Charm++ implementation [38] is encapsulated in a chare array. We can succinctly describe the parallel control flow of a non-pivoting LU in SDAG as follows: Each block goes through various phases as it executes depending on its location in the matrix.…”

“…In our highly-scalable implementation of dense LU [38], we demonstrate how to exploit Charm++ groups to control incoming messages by explicitly scheduling when messages arrive. For LU, instead of sending the block of data when it is ready on the sender-side, we notify the receiver that the data is ready and allow the receiver to determine which blocks to request based on what is ready and the optimized schedule it has computed that adheres to the dependencies natural in an LU computation.…”

“…As described earlier, we have implemented a dense LU factorization library [38] in Charm++ that fully conforms to the HPC Challenge [11] specification. Our implementation is a fully-composable library (it can share space and time with another parallel Charm++ module) that allows for flexible data placement (by writing a simple block-to-processor function).…”

Scalable Algorithms for Constructing Balanced Spanning Trees on System-Ranked Process Groups

“…The algorithms are then expressed by collectively addressing processes that own a row or column of matrix elements/blocks. For eg, recent work has demonstrated a high Terminology -n Number of processes in parent process group -m Number of processes participating in the new process group -k Branching factor (degree) of the spanning tree -d i,k Depth of a rank i process in a balanced spanning tree of branching factor k -f fraction of members of original process group participating in new group performance dense LU factorization using only the aforementioned collectives on non-trivially defined groups of processes, in a parallel programming paradigm that supports unranked and system-ranked process groups [10].…”

A Fine-Grained Pipelined Implementation of LU Decomposition on SIMD Processors

“…The task scheduling between CPUs and GPUs is also important. Optimized task scheduling can effectively reduce the computation [8][9][10][11][12], which is corresponding to the idea on general purpose processors [13]. Reconfigurable platforms always focus on designing high performance float-point computation units and the scalable architecture [14][15][16].…”