A High Throughput FPGA-Based Floating Point Conjugate Gradient Implementation

Abstract

Recent developments in the capacity of modern Field Programmable Gate Arrays (FPGAs) have significantly expanded their applications. One such field is the acceleration of scientific computation and one type of calculation that is commonplace in scientific computation is the solution of systems of linear equations. A method that has proven in software to be very efficient and robust for finding such solutions is the Conjugate Gradient (CG) algorithm. In this paper we present a widely-parallel and deeply-pipelined hardware CG implementation, targeted at modern FPGA architectures. This implementation is particularly suited for accelerating multiple small-to-medium sized dense systems of linear equations and can be used as a stand alone solver or as building block to solve higher order systems. In this paper it is shown that through parallelization it is possible to convert the computation time per iteration for an order n matrix from Θ(n 2 ) clock cycles on a micro-processor to Θ(n) on a FPGA. Through deep-pipelining it is also possible to solve several problems in parallel and maximize both performance and efficiency. I/O requirements are shown to be scalable and convergent to a constant value with the increase of matrix order. Post place-and-route results on a readily available VirtexII-6000 demonstrate sustained performance of 5 GFLOPS, and results on a Virtex5-330 indicate sustained performance of 35 GFLOPS. A comparison with an optimized software implementation running on a high-end CPU, demonstrate that this FPGA implementation represents a significant speed-up of at least an order of magnitude.

Model predictive control for deeply pipelined field-programmable gate array implementation: algorithms and circuitry

“…However, the benefits can only be attained by using factorization-based methods for solving linear systems, since the factorization is only computed once for both systems. Previous work [18], [19] suggests that iterative methods might be preferable in an FPGA implementation, due to the small number of division operations, which are very expensive in hardware, and because they allow one to trade off accuracy for computation time. In addition, these methods are easy to parallelize since they mostly consist of large matrix-vector multiplications.…”

“…The overall latency of the circuit will be given by Latency = 2I NW (P N + P )I MR FPGA f req seconds, (18) where I NW is the number of outer iterations in the interior-point method (Algorithm 1), FPGA f req is the FPGA's clock frequency, and P is given by (15). In that time the controller will be able to output the result to 2P problems.…”

“…Expression (18) consists of quadratic, linear and constant terms with respect to the number of inputs. If m is small compared to n and T , the constant term dominates and the improvement from using multiplexed MPC diminishes as a consequence.…”

A High Throughput FPGA-Based Implementation of the Lanczos Method for the Symmetric Extremal Eigenvalue Problem

“…We can accelerate scientific computations like the solution to systems of linear equations [2], [3] and [4] due to ever-increasing capacity of modern FPGAs in terms of floating point units and on-chip memory. The symmetric extremal eigenvalue problem is an important scientific computation involving dense linear algebra where one is interested in finding only the extremal eigenvalues of a n × n symmetric matrix.…”

An efficient sparse conjugate gradient solver using a Beneš permutation network

“…end for 14: end function CG implementations for dense systems [4] are easier to pipeline and achieve considerably higher memory efficiency due to the regular memory access pattern. On the other hand, sparse CG (where the system matrix A is sparse) is harder to parallelise due to the irregular access pattern and the mixture of both sparse and dense operations.…”