Compiling Generalized Histograms for GPU

Henriksen, Troels; Hellfritzsch, Sune; Sadayappan, P.; Oancea, Cosmin E.

doi:10.1109/sc41405.2020.00101

Cited by 11 publications

(12 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…5.1.2 Reduce-by-Index. Reduce-by-index [17], a.k.a. multireduce, essentially generalizes a histogram computation by allowing the values from an array ( ) that fall into the same bin (index from inds) to be reduced with an arbitrary associative and commutative operator ⊙ having neutral element ⊙ , where the number of bins is typically assumed smaller than that of index-value pairs , i.e., let hs = reduce_by_index (⊙) ⊙ inds has the semantics:…”

Section: Reduce and Multi-reducementioning

confidence: 99%

“…Another shortcoming is that when an in-place update occurs inside an if-then-else, we currently save the target array at the branch's entry: in principle, we should instead propagate the saved element(s)-denoted in Section 5.3outside branches, and back in to reach the return sweep. 17 By construction, this propagation cannot escape the scope of the innermost-enclosing recurrence. 1.…”

Section: Current Implementation Limitationsmentioning

confidence: 99%

“…-means clustering can also be solved using a method based on repeatedly calculating histograms [17], which can be implemented using Futhark's dedicated histogram construct. The code that is generated by AD consists of maps that perform semantically equivalent histogram-like accumulator updates.…”

Section: Compared To Enzymementioning

confidence: 99%

“…In particular, the simplest parallel operator, namely map, is the most difficult one to translate, because its purely functional semantics allows free variables to be freely read inside it, but reverse AD replaces a read with an accumulation, which cannot be in general represented as a combination of classical data-parallel constructs. In this context we report optimizations related to turning accumulators to more-specialized constructs, such as reductions and generalized histograms [17], that can yield speedups close to one order of magnitude at the application level (e.g., GMM and LSTM).…”

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

AD for an Array Language with Nested Parallelism

Schenck¹,

Rønning²,

Henriksen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We present a technique for applying (forward and) reversemode automatic differentiation (AD) on a non-recursive secondorder functional array language that supports nested parallelism and is primarily aimed at efficient GPU execution.The key idea is to eliminate the need for a "tape" by relying on redundant execution to bring into each new scope all program variables that may be needed by the differentiated code. Efficient execution is enabled by the observation that perfectly-nested scopes do not introduce re-execution, and such perfect nests are produced by known compiler transformations, e.g., flattening. Our technique differentiates loops and bulk-parallel operators-such as map, reduce, histogram, scan, scatter-by specific rewrite rules, and aggressively optimizes the resulting nested-parallel code. We report an experimental evaluation that compares with established AD solutions and demonstrates competitive performance on nine common benchmarks from recent applied AD literature.

show abstract

Section: Reduce and Multi-reducementioning

confidence: 99%

Section: Current Implementation Limitationsmentioning

confidence: 99%

Section: Compared To Enzymementioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

AD for an Array Language with Nested Parallelism

Schenck¹,

Rønning²,

Henriksen³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Futhark [11,18] uses a conventional functional syntax. Futhark programs are written as a potentially-nested composition of second-order array combinators (SOACs) that have inherently-parallel semantics-such as map, reduce, scan, scatter, generalized histograms [17]-and loops that are always executed sequentially. Loops have the semantics of a tail recursive function, and they explicitly declare the variables that are variant throughout the execution of the loop.…”

Section: Incremental Flatteningmentioning

confidence: 99%

Dataset Sensitive Autotuning of Multi-versioned Code Based on Monotonic Properties

Munksgaard

Breddam

Henriksen

et al. 2021

Lecture Notes in Computer Science

Self Cite

View full text Add to dashboard Cite

Functional languages allow rewrite-rule systems that aggressively generate a multitude of semantically-equivalent but differently-optimized code versions. In the context of GPGPU execution, this paper addresses the important question of how to compose these code versions into a single program that (near-)optimally discriminates them across different datasets. Rather than aiming at a general autotuning framework reliant on stochastic search, we argue that in some cases, a more effective solution can be obtained by customizing the tuning strategy for the compiler transformation producing the code versions.We present a simple and highly-composable strategy which requires that the (dynamic) program property used to discriminate between code versions conforms with a certain monotonicity assumption. Assuming the monotonicity assumption holds, our strategy guarantees that if an optimal solution exists it will be found. If an optimal solution doesn’t exist, our strategy produces human tractable and deterministic results that provide insights into what went wrong and how it can be fixed.We apply our tuning strategy to the incremental-flattening transformation supported by the publicly-available Futhark compiler and compare with a previous black-box tuning solution that uses the popular OpenTuner library. We demonstrate the feasibility of our solution on a set of standard datasets of real-world applications and public benchmark suites, such as Rodinia and FinPar. We show that our approach shortens the tuning time by a factor of $$6\times $$ 6 × on average, and more importantly, in five out of eleven cases, it produces programs that are (as high as $$10\times $$ 10 × ) faster than the ones produced by the OpenTuner-based technique.

show abstract