Intrepydd: performance, productivity, and portability for data science application kernels

Zhou, Tong; Shirako, Jun; Jain, Anirudh; Srikanth, Sriseshan; Conte, Thomas M.; Vuduc, Richard; Sarkar, Vivek

doi:10.1145/3426428.3426915

Cited by 7 publications

(4 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The Intrepydd programming language [23] introduced a subset of Python that is amenable to ahead-of-time (AOT) compilation into C++. It is intended for writing kernel functions rather than complete or main programs.…”

Section: Intrepydd Compilermentioning

confidence: 99%

“…The AutoMPHC compiler is the extension of Intrepydd compiler [23], which supports type inference and basic optimizations including loop invariant code motion, sparsity optimization, and array allocation/slicing optimizations. In the following sections, we present newly developed optimizations for automatic parallelization targeting distributed heterogeneous systems.…”

Section: Optimizationsmentioning

confidence: 99%

“…For example, even though Python bindings for MPI [9] have been available for many years, there has been very little adoption of these bindings by domain scientists. An alternate approach is to augment a highproductivity language with native libraries that include high-performance implementations of commonly used functions, e.g., functions in the NumPy [2] and SciPy [1] libraries for Python. However, fixed library interfaces and implementations do not address the needs of new applications and algorithms.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Automatic Parallelization of Python Programs for Distributed Heterogeneous Computing

Shirako¹,

Hayashi²,

Paul³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

This paper introduces a novel approach to automatic aheadof-time (AOT) parallelization and optimization of sequential Python programs for execution on distributed heterogeneous platforms. Our approach enables AOT source-to-source transformation of Python programs, driven by the inclusion of type hints for function parameters and return values. These hints can be supplied by the programmer or obtained by dynamic profiler tools; multi-version code generation guarantees the correctness of our AOT transformation in all cases. Our compilation framework performs automatic parallelization and sophisticated high-level code optimizations for the target distributed heterogeneous hardware platform. It includes extensions to the polyhedral framework that unify user-written loops and implicit loops present in matrix/tensor operators, as well as automated section of CPU vs. GPU code variants. Further, our polyhedral optimizations enable both intranode and inter-node parallelism. Finally, the optimized output code is deployed using the Ray runtime for scheduling distributed tasks across multiple heterogeneous nodes in a cluster. Our empirical evaluation shows significant performance improvements relative to sequential Python in both single-node and multi-node experiments, with a performance improvement of over 20,000× when using 24 nodes and 144 GPUs in the OLCF Summit supercomputer for the Space-Time Adaptive Processing (STAP) radar application.

show abstract

Section: Intrepydd Compilermentioning

confidence: 99%

Section: Optimizationsmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Automatic Parallelization of Python Programs for Distributed Heterogeneous Computing

Shirako¹,

Hayashi²,

Paul³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…As a result, frameworks for improving code portability have taken center stage as a means to reduce the amount of code refactoring effort necessary to facilitate the adoption of new hardware accelerators in HPC. Moreover, with the performance of general-purpose computing quickly plateauing, HPC software solutions must look toward more application-and domain-specific accelerators (ASAs/DSAs) to reach the next notable milestones in performance [25] [29]. This will undoubtedly increase the cadence of code refactoring to an impractical level with existing solutions, and more so, without.…”

Section: Introductionmentioning

confidence: 99%

FLASH 1.0: A Software Framework for Rapid Parallel Deployment and Enhancing Host Code Portability in Heterogeneous Computing

Riera¹,

Quraishi²,

Bank-Tavakoli³

et al. 2021

Preprint

View full text Add to dashboard Cite

In this paper, we present FLASH 1.0, a C++-based software framework for rapid parallel deployment and enhancing host code portability in heterogeneous computing. FLASH takes a novel approach in describing kernels and dynamically dispatching them in a hardwareagnostic manner. FLASH features truly hardware-agnostic frontend interfaces, which not only unify the compile-time control flow but also enforces a portability-optimized code organization that imposes a demarcation between computational (performance-critical) and functional (non-performance-critical) codes as well as the separation of hardware-specific and hardware-agnostic codes in the host application. We use static code analysis to measure the hardware independence ratio of popular HPC applications and show that up to 99.72% code portability can be achieved with FLASH. Similarly, we measure the complexity of state-of-the-art portable programming models and show that a code reduction of up to 2.2x can be achieved for two common HPC kernels while maintaining 100% code portability with a normalized framework overhead between 1% -13% of the total kernel runtime. The codes are available at https://github.com/PSCLab-ASU/FLASH. CCS CONCEPTS• Software and its engineering → Object oriented frameworks; Software as a service orchestration system.

show abstract

Automatic Parallelization of Python Programs for Distributed Heterogeneous Computing

Shirako

Hayashi

Paul

et al. 2022

Euro-Par 2022: Parallel Processing

View full text Add to dashboard Cite

Intrepydd: performance, productivity, and portability for data science application kernels

Cited by 7 publications

References 24 publications

Automatic Parallelization of Python Programs for Distributed Heterogeneous Computing

Automatic Parallelization of Python Programs for Distributed Heterogeneous Computing

FLASH 1.0: A Software Framework for Rapid Parallel Deployment and Enhancing Host Code Portability in Heterogeneous Computing

Automatic Parallelization of Python Programs for Distributed Heterogeneous Computing

Contact Info

Product

Resources

About