Flexible Development of Dense Linear Algebra Algorithms on Massively Parallel Architectures with DPLASMA

Abstract

We present a method for developing dense linear algebra algorithms that seamlessly scales to thousands of cores. It can be done with our project called DPLASMA (Distributed PLASMA) that uses a novel generic distributed Direct Acyclic Graph Engine (DAGuE). The engine has been designed for high performance computing and thus it enables scaling of tile algorithms, originating in PLASMA, on large distributed memory systems. The underlying DAGuE framework has many appealing features when considering distributed-memory platforms with heterogeneous multicore nodes: DAG representation that is independent of the problem-size, automatic extraction of the communication from the dependencies, overlapping of communication and computation, task prioritization, and architecture-aware scheduling and management of tasks. The originality of this engine lies in its capacity to translate a sequential code with nested-loops into a concise and synthetic format which can then be interpreted and executed in a distributed environment. We present three common dense linear algebra algorithms from PLASMA (Parallel Linear Algebra for Scalable Multi-core Architectures), namely: Cholesky, LU, and QR factorizations, to investigate their data driven expression and execution in a distributed system. We demonstrate through experimental results on the Cray XT5 Kraken system that our DAG-based approach has the potential to achieve sizable fraction of peak performance which is characteristic of the state-of-the-art distributed numerical software on current and emerging architectures.

From Serial Loops to Parallel Execution on Distributed Systems

“…For every parameter that appears in the execution space of a Relation's destination, we solve the equality constraints in the conjunction of constraints for this parameter. Consider, as an example, the Relation: This way, when the run-time is processing task T b (7,8) for example, it can compute in O(1) time that it needs to send tile A [8][8] to task T a (8). Also, when processing task T b (7,11), the run-time can compute that A[11] [11] should not be sent to any instance of T a , since the condition (1 + k) == m is not true (clearly, 1 + 7 = 11).…”

“…be the parameters of the task that correspond to N0 /* Initiate Cycle(N0) with an empty (tautologic) */ /* Relation to self. The performance of the DAGuE run-time has been extensively studied in related publications [8,9,7]. The goal of this paper is to present the compiler front-end of the system, so we present only a summary of performance results to demonstrate that our toolchain can automatically analyze, schedule and execute non-trivial algorithms, and deliver high performance at scale.…”

Scalable and Flexible heterogeneous multi-core system

“…[4] This architecture using many copies of the same core due to this improved total computational facility on single chip.Multi-core processors have enhanced performance and area characteristics than difficult single-core processors. [5] They propose and evaluate single-ISA heterogeneous multicore architectures as a system to reduce processor power dissipation.…”

“…Multi-core processors have improved performance and area characteristics than complex single-core processors. [5] Assess single-ISA heterogeneous multi-core architectures as a method to decrease processor power dissipation. www.ijacsa.thesai.org…”

Design and Implementation of the PULSAR Programming System for Large Scale Computing

“…Two notable projects in this category are Charm++ [36] and PaRSEC [9], which deal with algorithms and their implementation represented as a Direct Acyclic Graph (DAG) of tasks connected with edges that communicate data between them -a concept clearly related to the dataflow paradigm. Many other systems offer similar paradigm but might not afford the same type of support for distributed memory parallelism [5,47].…”