No abstract

Layout-Aware Data Scheduler (LADS) data transfer tool, identifies and addresses the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environments. It exploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS can avoid congested storage elements within the shared storage resource, improving input/output bandwidth, and hence the data transfer rates across the high speed networks. However, absence of FT (fault tolerance) support in LADS results in data retransmission overhead along with the possible integrity issues upon errors. In this paper, we propose object based logging methods to avoid transmitting the objects which are successfully written to Parallel File System (PFS) at the sink end. Depending on the number of logger files created, for the whole dataset, we classified our fault tolerance mechanisms into three different categories: File logger, Transaction logger and Universal logger. Also, to address space overhead of these object based logging mechanisms, we have proposed * Corresponding author Author(s) declare(s) that there is no conflict of interest regarding the publication of this paper.

No abstract