Stateful dataflow multigraphs

Abstract

The ubiquity of accelerators in high-performance computing has driven programming complexity beyond the skill-set of the average domain scientist. To maintain performance portability in the future, it is imperative to decouple architecture-specific programming paradigms from the underlying scientific computations. We present the Stateful DataFlow multiGraph (SDFG), a data-centric intermediate representation that enables separating program definition from its optimization. By combining fine-grained data dependencies with high-level control-flow, SDFGs are both expressive and amenable to program transformations, such as tiling and double-buffering. These transformations are applied to the SDFG in an interactive process, using extensible pattern matching, graph rewriting, and a graphical user interface. We demonstrate SDFGs on CPUs, GPUs, and FPGAs over various motifs -from fundamental computational kernels to graph analytics. We show that SDFGs deliver competitive performance, allowing domain scientists to develop applications naturally and port them to approach peak hardware performance without modifying the original scientific code.HPC programmers have long sacrificed ease of programming and portability for achieving better performance. This mindset was established at a time when computer nodes had a single processor/core and were programmed with C/Fortran and MPI. The last decade, witnessing the end of Dennard scaling and Moore's law, brought a flurry of new technologies into the compute nodes. Those range from simple multi-core and manycore CPUs to heterogeneous GPUs and specialized FPGAs. To support those architectures, the complexity of OpenMP's specification grew by more than an order of magnitude from 63 pages in OpenMP 1.0 to 666 pages in OpenMP 5.0. This one example illustrates how (performance) programming complexity shifted from network scalability to node

Reducing FPGA Memory Footprint of Stencil Codes through Automatic Extraction of Memory Patterns

“…We verify that our reduced approach achieves better performance-per-Watt compared to the original Fortran code running on a single CPU 2 .…”

“…We have presented an algorithm to remove intermediate arrays from stencil codes. We have incorporated this algorithm 2 Intel® Xeon® Platinum 8260, with GNU Fortran (GCC) 4.8.5 with -Ofast in our compilation toolchain from legacy Fortran code to SYCL, with the effect of significantly reducing FPGA memory resource usage. Our compiler expresses dataflow, parallelism and stencil accesses in a functional, domain-specific language (TyTraCL).…”

“…The DaCe framework implements code transformations at the dataflow level, generating HLS code for FPGAs [2]. DaCe programs are usually written in Python/NumPy, with their data flow between side-effect free functions represented as a flowbased Stateful Dataflow Multigraph (SDFG).…”

Upgrade ofD+ software for hierarchical modeling of X-ray scattering data from complex structures in solution, fibers and single orientations

“…The following subsections are based on the functions inside the g(r) module, whose functions work, where relevant, with NumPy arrays (Harris et al, 2020). The structure factor and the radial distribution function which are based on a structural model [equations ( 4)-( 6)] have been parallelized on CPUs using the DaCe algorithm (Ben-Nun et al, 2019). The parallel computation allowed us to get results in a reasonable time, even for large models.…”

Myths and legends in high-performance computing

“…Then the question remains—where should the portability layer be located? At a (virtualized) Instruction Set Architecture (ISA) as in LLVM’s IR (Lattner and Adve 2004), some lower-level language such as C/C++ as in SYCL/oneAPI, or even dataflow graph representations as in DaCe (Ben-Nun et al, 2019)?…”