Optimistic Parallelization of Floating-Point Accumulation

Abstract

Abstract-Floating-point arithmetic is notoriously nonassociative due to the limited precision representation which demands intermediate values be rounded to fit in the available precision. The resulting cyclic dependency in floating-point accumulation inhibits parallelization of the computation, including efficient use of pipelining. In practice, however, we observe that floating-point operations are "mostly" associative. This observation can be exploited to parallelize floating-point accumulation using a form of optimistic concurrency. In this scheme, we first compute an optimistic associative approximation to the sum and then relax the computation by iteratively propagating errors until the correct sum is obtained. We map this computation to a network of 16 statically-scheduled, pipelined, double-precision floating-point adders on the Virtex-4 LX160 (-12) device where each floating-point adder runs at 296 MHz and has a pipeline depth of 10. On this 16 PE design, we demonstrate an average speedup of 6× with randomly generated data and 3-7× with summations extracted from Conjugate Gradient benchmarks.

Synthesis of Floating-Point Addition Clusters on FPGAs Using Carry-Save Arithmetic

“…Kapre and DeHon [8] developed a speculative approach to parallel floating-point accumulation that maintains compliance. Operands are added two at a time, in parallel, using IEEE-754-compliant operators under the optimistic assumption that all operators are associative for the given inputs.…”

IEEE-754 compliant Algorithms for Fast Multiplication of Double Precision Floating Point Numbers

“…There are many multiplier architectures using different multiplication techniques for example, Array multiplier, redundant binary architecture and many more architectures using tree structures but they have problem of larger delay [1][2][3][4][5]. Different algorithms are also there that perform floating point multiplication like BOOTH algorithm, algorithms based on ancient mathematics [11], and many more. Ali Akoglu introduced an algorithm using ancient Vedic mathematics rules for fast multiplication which generate partial products concurrently.…”

“…As shown in Figure-1. Then accumulation is done of partial products and then finally partial products are added using binary tree [5], [7], [8], [11]. Accumulation is done by using Wallace tree as shown in Figure2.…”

“…Let A & B is two binary numbers to be multiplied more [11]. And numbers of partial products are not reduced so adder units needed are more as compare to CSD.…”

Sparse matrix-vector multiplication on network-on-chip

“…For example, the fattree style designs, which required presorting and preordering before input the data (Kapre and DeHon, 2007), will become extremely difficult when the matrix is very large and sparse. This challenge brings current applications and technology trends to motivate a paradigm shift in on-chip interconnect architectures from bus-based point-to-point network to packet-based switch network.…”