A Case for Scaling HPC Metadata Performance through De-specialization

Patil, Swapnil; Ren, Kai; Gibson, Garth A.

doi:10.1109/sc.companion.2012.372

Cited by 10 publications

(5 citation statements)

References 6 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This is caused by complex distributed locking of central data structures (generally managed by a metadata server instance) that are required to be accessed in parallel [31]. The file system community presented various techniques for handling metadata [12,32,35,36,37,38], but this challenge is still prevailing and is becoming an even bigger challenge for upcoming data-science applications.…”

Section: Related Workmentioning

confidence: 99%

GekkoFS — A Temporary Burst Buffer File System for HPC Applications

Vef

Moti

Süß

et al. 2020

J. Comput. Sci. Technol.

View full text Add to dashboard Cite

Many scientific fields increasingly use High-Performance Computing (HPC) to process and analyze massive amounts of experimental data while storage systems in today's HPC environments have to cope with new access patterns. These patterns include many metadata operations, small I/O requests, or randomized file I/O, while general-purpose parallel file systems have been optimized for sequential shared access to large files. Burst buffer file systems create a separate file system that applications can use to store temporary data. They aggregate node-local storage available within the compute nodes or use dedicated SSD clusters and offer a peak bandwidth higher than that of the backend parallel file system without interfering with it. However, burst buffer file systems typically offer many features that a scientific application, running in isolation for a limited amount of time, does not require. We present GekkoFS, a temporary, highly-scalable file system which has been specifically optimized for the aforementioned use cases. GekkoFS provides relaxed POSIX semantics which only offers features which are actually required by most (not all) applications. GekkoFS is, therefore, able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of common parallel file systems.

show abstract

Section: Related Workmentioning

confidence: 99%

GekkoFS — A Temporary Burst Buffer File System for HPC Applications

Vef

Moti

Süß

et al. 2020

J. Comput. Sci. Technol.

View full text Add to dashboard Cite

show abstract

“…Typically, general-purpose PFSs distribute data across all available storage targets. As this technique works well for data, it does not achieve the same throughput when handling metadata [5], [28], although the file system community presented various techniques to tackle this challenge [3], [13], [25], [26], [41], [42]. The performance limitation can be attributed to the sequentialization enforced by underlying POSIX semantics which is particularly degrading throughput when a huge number of files is created in a single directory from multiple processes.…”

Section: Related Workmentioning

confidence: 99%

GekkoFS - A Temporary Distributed File System for HPC Applications

Vef

Moti

SuB³

et al. 2018

2018 IEEE International Conference on Cluster Computing (CLUSTER)

View full text Add to dashboard Cite

We present GekkoFS, a temporary, highly-scalable burst buffer file system which has been specifically optimized for new access patterns of data-intensive High-Performance Computing (HPC) applications. The file system provides relaxed POSIX semantics, only offering features which are actually required by most (not all) applications. It is able to provide scalable I/O performance and reaches millions of metadata operations already for a small number of nodes, significantly outperforming the capabilities of general-purpose parallel file systems.

show abstract

“…FPFS also implements management for huge directories, or hugedirs for short. These directories are common for some HPC applications, such as those that create a file per thread/process, and those that use a directory as a light‐weight database (eg, check pointing) . To manage hugedirs, FPFS proposes a dynamic distribution of their entries among multiple OSD+s .…”

Section: Overview Of Fpfsmentioning

confidence: 99%

Leveraging OSD+ devices for implementing a high‐throughput parallel file system

Piernas

González-Férez

2018

Concurrency and Computation

View full text Add to dashboard Cite

OSD+s are enhanced object-based storage devices (OSDs) able to deal with both data and metadata operations via data and directory objects, respectively. So far, we have focused on designing and implementing efficient directory objects in OSD+s. This paper, however, presents our work on also supporting data objects and describes how the coexistence of both kinds of objects in each OSD+ is profited to efficiently implement data objects and to speed up some common file operations. We compare our OSD+-based Fusion Parallel File System (FPFS) with Lustre and OrangeFS through different microbenchmarks and HPCS-IO scenarios. Results show that FPFS provides a throughput up to 37× better than Lustre and up to 95× better than OrangeFS for metadata workloads. FPFS also provides 34% more bandwidth than OrangeFS for data workloads and competes with Lustre in data writes. Results also show serious scalability problems in Lustre and OrangeFS that limit their performance. KEYWORDS data objects, FPFS, Lustre, OrangeFS, OSD+ INTRODUCTIONFile systems for HPC environments have traditionally used a cluster of data servers for achieving high rates in read and write operations, fault tolerance, scalability, etc. However, due to a growing number of files and an increasing use of huge directories with millions or billions of entries accessed by thousands of clients at the same time, 1-3 some of these file systems also utilize a cluster of specialized metadata servers 4-6 and have recently added support for distributed directories. 5,7Unlike those file systems that have separate data and metadata clusters, our in-house Fusion Parallel File System (FPFS) uses a single cluster of object-based storage device+ (OSD+) 8 to implement those clusters. OSD+s are improved object-based storage devices (OSDs) that, in addition to handle data objects as traditional OSDs do, can also manage directory objects. These objects are a new type of object able to store file names and attributes, and support metadata-related operations. By using these OSD+ devices, an FPFS metadata cluster is as large as its corresponding data cluster, and metadata is effectively distributed among as many nodes as OSD+s comprising the system. OSD+s are implemented through a thin software layer on top of existing mainstream computers, leveraging many features of the underlying file system. FPFS also supports huge directories by dynamically distributing them among several OSD+s. 9 The OSD+s storing a distributed huge directory work independently of each other, thereby improving the performance and scalability of the file system.So far, we have focused on the development of the metadata part of FPFS. In this paper, however, we describe how we have implemented the support for data objects. We show that the utilization of a unified data and metadata server (ie, an OSD+ device) provides FPFS with a competitive advantage with respect to other file systems that allows it to speed up some file operations, such as creating and deleting files and getting the status of files.We evaluate th...

show abstract

A Case for Scaling HPC Metadata Performance through De-specialization

Cited by 10 publications

References 6 publications

GekkoFS — A Temporary Burst Buffer File System for HPC Applications

GekkoFS — A Temporary Burst Buffer File System for HPC Applications

GekkoFS - A Temporary Distributed File System for HPC Applications

Leveraging OSD+ devices for implementing a high‐throughput parallel file system

Contact Info

Product

Resources

About