A method for reliably managing files with RNS in multi Data Grids

Abstract

This paper describes a method for reliably managing files distributed in different kinds of Data Grids with RNS (Resource Namespace Service). RNS provides hierarchical namespace management for name-to-resource mapping as a key technology when using Grid resources for different kinds of middleware. We define attribute expressions in XML for the RNS entries and give algorithms to access distributed files stored within different kinds of Data Grids.The volume of digital data and the size of an individual file are increasing due to the introduction of high-resolution images, high-definition audiovisual files, etc. The reliable storage of such large files is becoming problematic with whole file replication as a failure in the integrity of the file is difficult to localise. Our method involves managing large files in Data Grids by splitting them into smaller units in a traceable manner and then managing the smaller units. The RNS catalog service contains EPR (Endpoint Reference) and metadata that describe the original locations as well as the checksum values. The example in this paper shows how our Grid application can retrieve the actual file locations and the checksum values from the RNS service via SAGA (A Simple API for Grid Applications). An application can access the distributed files as though they were files in the local file-system without worrying about the underlying Data Grids.This approach can be used with various Data Grid systems to enhance file reliability.

Universal Grid User Interface (UGI) for Multiple Grids and Clouds

“…The SAGA API facilitates application developers to build real applications used by the end user of scientists. Our developments of such adaptors enabled to show how to use resources distributed in different Grids [16] and solves several problems [17,18,19].…”