Data flow fusion with series expressions in Haskell

Lippmeier, Ben; Chakravarty, Manuel M. T.; Keller, Gabriele; Robinson, Amos

doi:10.1145/2503778.2503782

Cited by 10 publications

(8 citation statements)

References 27 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Lippmeier et al [18] present a line of work based on SERIES. They aim to transform first-order, non-recursive, synchronous, finite data-flow programs into fused pipelines.…”

Section: Related Workmentioning

confidence: 99%

Stream fusion, to completeness

KiselyovOleg¹,

BiboudisAggelos²,

PalladinosNick³

et al. 2017

SIGPLAN Not.

View full text Add to dashboard Cite

Stream processing is mainstream (again): Widely-used stream libraries are now available for virtually all modern OO and functional languages, from Java to C# to Scala to OCaml to Haskell. Yet expressivity and performance are still lacking. For instance, the popular, well-optimized Java 8 streams do not support the zip operator and are still an order of magnitude slower than hand-written loops. We present the first approach that represents the full generality of stream processing and eliminates overheads, via the use of staging. It is based on an unusually rich semantic model of stream interaction. We support any combination of zipping, nesting (or flat-mapping), sub-ranging, filtering, mapping—of finite or infinite streams. Our model captures idiosyncrasies that a programmer uses in optimizing stream pipelines, such as rate differences and the choice of a “for” vs. “while” loops. Our approach delivers hand-written–like code, but automatically. It explicitly avoids the reliance on black-box optimizers and sufficiently-smart compilers, offering highest, guaranteed and portable performance. Our approach relies on high-level concepts that are then readily mapped into an implementation. Accordingly, we have two distinct implementations: an OCaml stream library, staged via MetaOCaml, and a Scala library for the JVM, staged via LMS. In both cases, we derive libraries richer and simultaneously many tens of times faster than past work. We greatly exceed in performance the standard stream libraries available in Java, Scala and OCaml, including the well-optimized Java 8 streams.

show abstract

“…Lippmeier et al [18] present a line of work based on SERIES. They aim to transform first-order, non-recursive, synchronous, finite data-flow programs into fused pipelines.…”

Section: Related Workmentioning

confidence: 99%

Stream fusion, to completeness

KiselyovOleg¹,

BiboudisAggelos²,

PalladinosNick³

et al. 2017

SIGPLAN Not.

View full text Add to dashboard Cite

show abstract

“…This relation arises naturally from Data Flow Fusion [12], as such an operation would be compiled into a single loop -with an iteration size identical to the size of the input array, and containing two nested if-expressions to perform the two layers of filtering.…”

Section: Iteration Sizementioning

confidence: 99%

“…Although we also have an implementation of our Data Flow Fusion system in terms of a GHC plugin [12], we report on hand-fused C code to provide a fair comparison to related work. As mentioned in [12], the current Haskell stream fusion mechanism introduces overhead in terms of a large number of duplicate loop counters, which increases register pressure unnecessarily. Hand fusing all code and compiling it with the same compiler (GCC) isolates the true cost of the various clusterings from low level differences in code generation.…”

Section: Benchmarksmentioning

confidence: 99%

“…To compile the clusters found by our clustering technique into sequential loops, we use data flow fusion [12]. It improved on existing array fusion approaches [5,10] as it guarantees fusion into Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Fusing filters with integer linear programming

Robinson

Lippmeier

Keller

2014

Proceedings of the 3rd ACM SIGPLAN Workshop on Functional High-Performance Computing

Self Cite

View full text Add to dashboard Cite

The key to compiling functional, collection oriented array programs into efficient code is to minimise memory traffic. Simply fusing subsequent array operations into a single computation is not sufficient; we also need to cluster separate traversals of the same array into a single traversal. Previous work demonstrated how Integer Linear Programming (ILP) can be used to cluster the operators in a general data-flow graph into subgraphs, which can be individually fused. However, these approaches can only handle operations which preserve the size of the array, thereby missing out on some optimisation opportunities. This paper addresses this shortcoming by extending the ILP approach with support for size-changing operations, using an external ILP solver to find good clusterings.

show abstract

“…The resulting functions of high arity create register pressure, which has performance implications. A newly developed Flow Fusion system [16], based on Series Expressions [31], attempts to address this problem. However, Flow Fusion targets static, first-order, non-recursive stream pipelines, which necessarily excludes concatM ap.…”

Section: Related Workmentioning

confidence: 99%

The HERMIT in the stream

Farmer

Siederdissen

Gill

2014

Proceedings of the ACM SIGPLAN 2014 Workshop on Partial Evaluation and Program Manipulation

View full text Add to dashboard Cite

Stream Fusion, a popular deforestation technique in the Haskell community, cannot fuse the concatM ap combinator. This is a serious limitation, as concatM ap represents computations on nested streams. The original implementation of Stream Fusion used the Glasgow Haskell Compiler's user-directed rewriting system. A transformation which allows the compiler to fuse many uses of concatM ap has previously been proposed, but never implemented, because the host rewrite system was not expressive enough to implement the proposed transformation. In this paper, we develop a custom optimization plugin which implements the proposed concatM ap transformation, and study the effectiveness of the transformation in practice. We also provide a new translation scheme for list comprehensions which enables them to be optimized. Within this framework, we extend the transformation to monadic streams. Code featuring uses of concatM ap experiences significant speedup when compiled with this optimization. This allows Stream Fusion to outperform its rival, foldr/build, on many list computations, and enables performance-sensitive code to be expressed at a higher level of abstraction.

show abstract

Data flow fusion with series expressions in Haskell

Cited by 10 publications

References 27 publications

Stream fusion, to completeness

Stream fusion, to completeness

Fusing filters with integer linear programming

The HERMIT in the stream

Contact Info

Product

Resources

About