Noncontiguous locking techniques for parallel file systems

Abstract

Many parallel scientific applications use high-level I/O APIs that offer atomic I/O capabilities. Atomic I/O in current parallel file systems is often slow when multiple processes simultaneously access interleaved, shared files. Current atomic I/O solutions are not optimized for handling noncontiguous access patterns because current locking systems have a fixed file system block-based granularity and do not leverage highlevel access pattern information.In this paper we present a hybrid lock protocol that takes advantage of new list and datatype byte-range lock description techniques to enable high performance atomic I/O operations for these challenging access patterns. We implement our scalable distributed lock manager (DLM) in the PVFS parallel file system and show that these techniques improve locking throughput over a naive noncontiguous locking approach by several orders of magnitude in an array of lockonly tests. Additionally, in two scientific I/O benchmarks, we show the benefits of avoiding false sharing with our byterange granular DLM when compared against a block-based lock system implementation.

Conflict Detection Algorithm to Minimize Locking for MPI-IO Atomicity

“…We want to show that a locking mechanism combined with our approach is effective in reducing the number of lock requests that will be issued for any lock server that needs communication and space on the server to be stored. Many scientific applications have patterns that would require hundred thousands of locks [10]. In Table 1, we show a simple comparison of the number of locks (whole file, byte-range and list) with and without using conflict detection.…”

“…This approach also limits the benefits of parallel I/O that can be gained, by unnecessarily serializing the accesses. To address these particular cases, [5] [10] propose to lock the exact non-contiguous regions within a byte range and maximize the concurrent I/O access.…”

Efficient Support for MPI-I/O Atomicity Based on Versioning

“…Further optimizations are proposed in [10], where the authors propose a locking-based scheme for non-contiguous I/O which aims to strictly reduce the scope of the locked regions to the areas that are actually accessed. Moreover, this approach cannot avoid serialization for applications which exhibit concurrent overlapping I/O such as the ones described in Section 2.…”

“…We turned off data sieving in the default ROMIO ADIO module according to the recommendations in [10], as this greatly improves the performance of Lustre for MPI-IO. Without data sieving enabled, the ADIO module is able to take advantage of standard POSIX byte-range file locking to lock the smallest contiguous region in the file that covers all non-contigous regions that need to be read/written.…”

Interfaces for coordinated access in the file system

“…While support for advisory locks in many file systems has improved, they are by no means standardized. Several parallel file systems, most notably PVFS [21], do not support fcntl() locks at all. Lustre [22] supports fcntl() locks only if mounted with a special mount option.…”