Building Hierarchical Grid Storage Using the Gfarm Global File System and the JuxMem Grid Data-Sharing Service

Abstract

Abstract. As more and more large-scale applications need to generate and process very large volumes of data, the need for adequate storage facilities is growing. It becomes crucial to efficiently and reliably store and retrieve large sets of data that may be shared at the global scale. Based on previous systems for global data sharing (global file systems, grid data-sharing services), this paper proposes a hierarchical approach for grid storage, which combines the access efficiency of RAM storage with the scalability and persistence of the global file system approach. Our proposal has been validated through a prototype that couples the GFARM file system with the JUXMEM data-sharing service. Experiments on the Grid'5000 testbed confirm the advantages of our approach.

Enabling lock-free concurrent fine-grain access to massive distributed data: Application to supernovae detection

“…As opposed to grid file systems, our service mainly relies on RAM storage. This favors access efficiency, while data persistence can still be provided following the scheme described in [12]. Our paper is organized as follows.…”