Functional and dynamic programming in the design of parallel prefix networks

Abstract

A parallel prefix network of width n takes n inputs, a 1 , a 2 , . . . , a n , and computes each y i = a 1 • a 2 • · · · • a i for 1 6 i 6 n, for an associative operator •. This is one of the fundamental problems in computer science, because it gives insight into how parallel computation can be used to solve an apparently sequential problem. As parallel programming becomes the dominant programming paradigm, parallel prefix or scan is proving to be a very important building block of parallel algorithms and applications. There are many different parallel prefix networks, with different properties such as number of operators, depth and allowed fanout from the operators. In this paper, ideas from functional programming are combined with search to enable a deep exploration of parallel prefix network design. Networks that improve on the best known previous results are generated. It is argued that precise modelling in a functional programming language, together with simple visualization of the networks, gives a new, more experimental, approach to parallel prefix network design, improving on the manual techniques typically employed in the literature. The programming idiom that marries search with higher order functions may well have wider application than the network generation described here.

A sound and complete abstraction for reasoning about parallel prefix sums

“…An array of length n is processed by n/2 threads, except in the case of Kogge-Stone where n threads are used. Sheeran confirms this claim when exploiting the result in later work on the design of prefix sum algorithms [33]. We refer to this as the Sheeran result.…”

“…A sequential prefix sum for inputs of length n Our second main contribution is an extension of our method and theoretical results to the case of barrier-synchronising data-parallel programs, the programming model of GPU kernels. This is a contribution over previous work on the correctness of parallel prefix sums [33,38] which applies to synchronous parallel hardware described as sequential HASKELL programs, but not to asynchronous concurrent programs. We show that if a data-parallel program implementing a generic prefix sum can be proved free from data races then correctness of the prefix sum can be established by running a single test case using the interval of summations monoid, as in the sequential case.…”

Combining execution pipelines to improve parallel implementation of HMMER on FPGA

“…In Java, a network topology is described in the form of loops and recursive function calls. Sheeran [35] proposed similar but more generalized approach for prefix network generation from abstract specification, by modeling prefix networks using functional programming language, Haskell. In our case, the network construction in Java is more practical, as the objective is solely to generate RTL descriptions for existing prefix networks, and to evaluate their performance on FPGA.…”

Synchronous digital circuits as functional programs

“…Sheeran [2011] has used her various platforms to explore different kinds of circuits and has shown that rapid feedback in the form of simulation, testing, and model checking is most valuable to the designer. Johnson and Bose [1997] and Seger et al [2005] make similar observations about their refinement efforts.…”

“…They have also been used to analyze many other combinational circuits, such as adders and multipliers [Axelsson 2003], and as a host for a sequential language much simpler than what we discuss in Section 3.1 [Claessen 2001, Chapter 6]. More recently, Sheeran [2005Sheeran [ , 2011 has developed techniques for context-sensitive circuit generators and optimisers using this system.…”