A backward/forward recovery approach for the preconditioned conjugate gradient method

Abstract

Several recent papers have introduced a periodic verification mechanism to detect silent errors in iterative solvers. Chen [PPoPP'13, has shown how to combine such a verification mechanism (a stability test checking the orthogonality of two vectors and recomputing the residual) with checkpointing: the idea is to verify every d iterations, and to checkpoint every c × d iterations. When a silent error is detected by the verification mechanism, one can rollback to and re-execute from the last checkpoint. In this paper, we also propose to combine checkpointing and verification, but we use algorithm-based fault tolerance (ABFT) rather than stability tests. ABFT can be used for error detection, but also for error detection and correction, allowing a forward recovery (and no rollback nor re-execution) when a single error is detected. We introduce an abstract performance model to compute the performance of all schemes, and we instantiate it using the preconditioned conjugate gradient algorithm. Finally, we validate our new approach through a set of simulations.

Analysis of Monte Carlo accelerated iterative methods for sparse linear systems

“…x = Hx + f, (2) where H = I − A and f = b. Assuming that the spectral radius (H) < 1, the solution to Equation 2 can be written in terms of a power series in H (Neumann series):…”

Sparse approximate solution of fitting surface to scattered points by MLASSO model

“…When the matrix is ill-posed, i.e., its condition number is huge, the convergence rate of the CG will be very slow. Under this case, the preconditioned CG [4,22,43] can be used instead to reduce the condition number of the coefficient matrix and improve the convergence speed. The second subproblem has a simple analytical solution based on soft-thresholding operator [17], that is…”

TwinPCG: Dual Thread Redundancy with Forward Recovery for Preconditioned Conjugate Gradient Methods

“…Online-ABFT [4] ABFT SpMxV [5], [6] TwinCG the general case very unstable when such faults occur. It is well studied that its convergence can not be guaranteed even for rounding off errors, let alone for transient errors which can affect more significant bits in memory.…”

“…The basic idea is to introduce additional checksums, and some additional computation, which may allow for detection and correction of faults in the matrix-vector product. The same idea was recently applied to the sparse matrix-vector product in the work of [5] and [6]; these contributions improve the resilience of iterative solvers like CG by focusing on the underlying sparse matrix-vector product. We refer to algorithm-based fault-tolerant versions of sparse matrix-vector product as ABFT SpMxV.…”

“…The entire mechanism of ABFT SpMxV can be considered an efficient redundancy in a single-threaded execution, which uses some extra space (checksums), and extra time (additional computations). There are strict limits to what existing ABFT SpMxV can do: the related contributions [5], [6] detect up to two faults, and correct up to 1 fault.…”