The HERMIT in the stream

Abstract

Stream Fusion, a popular deforestation technique in the Haskell community, cannot fuse the concatM ap combinator. This is a serious limitation, as concatM ap represents computations on nested streams. The original implementation of Stream Fusion used the Glasgow Haskell Compiler's user-directed rewriting system. A transformation which allows the compiler to fuse many uses of concatM ap has previously been proposed, but never implemented, because the host rewrite system was not expressive enough to implement the proposed transformation. In this paper, we develop a custom optimization plugin which implements the proposed concatM ap transformation, and study the effectiveness of the transformation in practice. We also provide a new translation scheme for list comprehensions which enables them to be optimized. Within this framework, we extend the transformation to monadic streams. Code featuring uses of concatM ap experiences significant speedup when compiled with this optimization. This allows Stream Fusion to outperform its rival, foldr/build, on many list computations, and enables performance-sensitive code to be expressed at a higher level of abstraction.

Stream fusion, to completeness

“…The application of HERMIT [6] to streams [7] fixes the shortcomings of the Coutts et al Stream Fusion [5] On the other hand, the flat_map interface only supports nested streams of a fixed structure-hence with the applicative rather than monadic interface. We can provide a more general flat_map with the continuation-passing interface for the mapping function, which then implements:…”