Leveraging OSD+ devices for implementing a high‐throughput parallel file system

Abstract

OSD+s are enhanced object-based storage devices (OSDs) able to deal with both data and metadata operations via data and directory objects, respectively. So far, we have focused on designing and implementing efficient directory objects in OSD+s. This paper, however, presents our work on also supporting data objects and describes how the coexistence of both kinds of objects in each OSD+ is profited to efficiently implement data objects and to speed up some common file operations. We compare our OSD+-based Fusion Parallel File System (FPFS) with Lustre and OrangeFS through different microbenchmarks and HPCS-IO scenarios. Results show that FPFS provides a throughput up to 37× better than Lustre and up to 95× better than OrangeFS for metadata workloads. FPFS also provides 34% more bandwidth than OrangeFS for data workloads and competes with Lustre in data writes. Results also show serious scalability problems in Lustre and OrangeFS that limit their performance. KEYWORDS data objects, FPFS, Lustre, OrangeFS, OSD+ INTRODUCTIONFile systems for HPC environments have traditionally used a cluster of data servers for achieving high rates in read and write operations, fault tolerance, scalability, etc. However, due to a growing number of files and an increasing use of huge directories with millions or billions of entries accessed by thousands of clients at the same time, 1-3 some of these file systems also utilize a cluster of specialized metadata servers 4-6 and have recently added support for distributed directories. 5,7Unlike those file systems that have separate data and metadata clusters, our in-house Fusion Parallel File System (FPFS) uses a single cluster of object-based storage device+ (OSD+) 8 to implement those clusters. OSD+s are improved object-based storage devices (OSDs) that, in addition to handle data objects as traditional OSDs do, can also manage directory objects. These objects are a new type of object able to store file names and attributes, and support metadata-related operations. By using these OSD+ devices, an FPFS metadata cluster is as large as its corresponding data cluster, and metadata is effectively distributed among as many nodes as OSD+s comprising the system. OSD+s are implemented through a thin software layer on top of existing mainstream computers, leveraging many features of the underlying file system. FPFS also supports huge directories by dynamically distributing them among several OSD+s. 9 The OSD+s storing a distributed huge directory work independently of each other, thereby improving the performance and scalability of the file system.So far, we have focused on the development of the metadata part of FPFS. In this paper, however, we describe how we have implemented the support for data objects. We show that the utilization of a unified data and metadata server (ie, an OSD+ device) provides FPFS with a competitive advantage with respect to other file systems that allows it to speed up some file operations, such as creating and deleting files and getting the status of files.We evaluate th...

Foreword to the Special Issue on Processors, Interconnects, Storage, and Caches for Exascale Systems

“…Piernas and González‐Férez address the scalability of file systems aimed to exascale systems. In particular, they describe how they have implemented the support for data objects in their previously proposed Fusion Parallel File System (FPFS).…”