A Diskless Checkpointing Algorithm for Cluster Architectures Applied to Geospatial Raster Data Processing

Abstract

In recent years, due to the increasing calculation demands for the massive spatial data analysis, the parallel computing based on high-performance computers has become an inevitable trend of geospatial raster data processing, such as digital terrain analysis (DTA for short), remote sensing interpretation and digital soil mapping. A key problem is how to design a fault-tolerant software to enhance the stability and robustness of scientific application. This paper presents an approach of failure recovery for distributed memory parallel computing. Furthermore, we adopt the master/slave programming model and present a framework of redundant master mode, which the failure occurring on the master node could not lead to a breakdown of the whole system. This approach schedules the failing task by dividing all the failing data into several partitions according to the calculating scale of failure. By means of the Fault-Tolerant Granularity Model, the scheduling algorithm can assign the failing task dynamically. Finally, taking example of digital terrain analysis, two experiments are discussed that based on the data size and the number of failures. Simulation results indicate that the proposed scheduling algorithm based on Fault-Tolerant Granularity Model achieves lower fault tolerance overhead than the rollback recovery scheme when several processors fail simultaneously.

Performance evaluation of parallel re-computing algorithm in different data distribution modes

“…8,10 Song et al implemented a fault-tolerant parallel computing algorithm adopting checkpointing technique for DTA. 11 This re-computing method adopting redundancy mechanism is deeply discussed in theory. 12 An initial performance analysis for parallel re-computation method is discussed in literature.…”