Distributing and Parallelizing Non-canonical Loops

Abstract

This work leverages an original dependency analysis to parallelize loops regardless of their form in imperative programs. Our algorithm distributes a loop into multiple parallelizable loops, resulting in gains in execution time comparable to state-of-the-art automatic source-to-source code transformers when both are applicable. Our graph-based algorithm is intuitive, language-agnostic, proven correct, and applicable to all types of loops. Importantly, it can be applied even if the loop iteration space is unknown statically or at compile time, or more generally if the loop is not in canonical form or contains loop-carried dependency. As contributions we deliver the computational technique, proof of its preservation of semantic correctness, and experimental results to quantify the expected performance gains. We also show that many comparable tools cannot distribute the loops we optimize, and that our technique can be seamlessly integrated into compiler passes or other automatic parallelization suites.

Read/write factorizable programs

“…This is (roughly) the study of measures or bounds on programs or computation states, such that only those programs having execution that terminates within the bound, are part of the characterization. This is a flourishing field, both for traditional programs (Bonfante et al, 2011;Aubert et al, 2022) and for non-deterministic models such as term rewriting systems (Avanzini et al, 2011(Avanzini et al, , 2015Avanzini & Moser, 2016).…”

pymwp: A Static Analyzer Determining Polynomial Growth Bounds

“…3.3. The name alludes to its implementation language, Python, which we selected for its flexibility and use in previous related works [4,19,20]. Our tool takes as input a path to a C program, and returns for each function it contains a growth bound-if at least one can be established-or a list of variable dependencies that may cause the exponential growth .…”

“…Our tool takes as input a path to a C program, and returns for each function it contains a growth bound-if at least one can be established-or a list of variable dependencies that may cause the exponential growth . 6 The pymwp development is open source [24] with releases published at Python Package Index (PyPI) [22], GitHub [24] and Zenodo [7]. A tool user guide is available at https://statycc.github.io/.github/pymwp/.…”

“…For this reason, it is conjectured that ICC systems could be used to achieve realistic complexity analysis [18, p. 16]. Our series of work [5,6] is testing this hypothesis, and resulted in the tool we present in this paper: pymwp is one of the first ICC-inspired applications, and the first mechanization of the specific technique it implements. Let us first exemplify what pymwp calculates.…”

“…By manual analysis, we can deduce that the variable values "growth bound" between the initial values X1, X2 and X3 (overloading initial values and parameter names) and their final values (denoted X1', X2' and X3'), omitting constants, is X1' = X1, X2' ≤ max(X1, X2) and X3' ≤ max(X3, X2 + X1). 4 Therefore, for all initial values, the value growth of the variable's value is bounded by a polynomial w.r.t. its initial values.…”