Algorithm-Based Checkpoint-Recovery for the Conjugate Gradient Method

Abstract

As computers reach exascale and beyond, the incidence of faults will increase. Solutions to this problem are an active research topic. We focus on strategies to make the preconditioned conjugate gradient (PCG) solver resilient against node failures, speci cally, the exact state reconstruction (ESR) method, which exploits redundancies in PCG. Reducing the frequency at which redundant information is stored lessens the runtime overhead. However, after the node failure, the solver must restart from the last iteration for which redundant information was stored, which increases recovery overhead. This formulation highlights the method's similarities to checkpoint-restart (CR). Thus, this method, which we call ESR with periodic storage (ESRP), can be considered a form of algorithm-based checkpointrestart. The state is stored implicitly, by exploiting redundancy inherent to the algorithm, rather than explicitly as in CR. We also minimize the amount of data to be stored and retrieved compared to CR, but additional computation is required to reconstruct the solver's state. In this paper, we describe the necessary modi cations to ESR to convert it into ESRP, and perform an experimental evaluation. We compare ESRP experimentally with previously-existing ESR and application-level in-memory CR. Our results con rm that the overhead for ESR is reduced signi cantly, both in the failure-free case, and if node failures are introduced. In the former case, the overhead of ESRP is usually lower than that of CR. However, CR is faster if node failures happen. We claim that these di erences can be alleviated by the implementation of more appropriate preconditioners.

Optimal Checkpointing Strategies for Iterative Applications

“…Pachajoa et al [31] compared exact state reconstruction (ESR) approach based on the method proposed by Chen [7] with the heuristic linear interpolation (LI) approach by Langou et al [28] and Agullo et al [1,2]. They later extended the ESR approach for protecting the PCG method against multiple and simultaneous node failures [32,33]. Altogether, fault-tolerance methods proposed to mitigate the impact of fail-stop errors in iterative applications are application-specific, and can only be applied to a particular class of iterative algorithms.…”

NVM-ESR: Using Non-Volatile Memory in Exact State Reconstruction of Preconditioned Conjugate Gradient

“…There is a generic strategy for identifying the state of an iterative linear algebra solver and for reconstructing the state upon recovery [14]. However, like prior work [16,17], we focus on the preconditioned conjugate gradient (PCG) solver, which solves the linear equation 𝐴𝑥 = 𝑏 for a symmetric positive definite matrix 𝐴 𝑛×𝑛 (see Algorithm 1).…”

“…ESRP [17] is a modification of ESR, where redundant copies are created every period to alleviate the networking overhead for each iteration. ESRP demonstrates a trade-off, where increasing the period of ESR decreases the runtime overhead, but increases the cost of discarding the iterations performed since the last storage stage was reached when recovery is required.…”

“…Finally, it is necessary to determine where the state is saved. In traditional systems, the state has to be saved in other cluster nodes [17], so that whenever a node fails, the surviving nodes send the state of the failed node to a spare node (Figure 1a).…”

“…Necessary conditions on solvers for the applicability of ESR [14,16,17] are: (1) The iterative algorithm performs a finite-term recurrence, and (2) The iterative algorithm involves a matrix-vector product. In particular, ESR was applied to the Preconditioned Conjugate Gradient (PCG) solver, solving the linear equation 𝐴𝑥 = 𝑏 for a symmetric positive definite matrix 𝐴 𝑛×𝑛 .…”