Middleware support for many-task computing

Abstract

While the I/O functions described in the MPI standard included shared file pointer support from the beginning, the performance and portability of these functions have been subpar at best. ROMIO [1], which provides the MPI-IO functionality for most MPI libraries, to this day uses a separate file to manage the shared file pointer. This file provides the shared location that holds the current value of the shared file pointer. Unfortunately, each access to the shared file pointer involves file lock management and updates to the file contents. Furthermore , support for shared file pointers is not universally available because few file systems support native shared file pointers [5] and a few file systems do not support file locks [3]. Application developers rarely use shared file pointers, even though many applications can benefit from this file I/O capability. These applications are typically loosely coupled and rarely exhibit application-wide synchronization. Examples include application tracing toolkits [8,4] and many-task computing applications [10]. Other approaches to the shared file pointer I/O models frequently used by these application classes include file-per-process, file-per-thread, and file-per-rank approaches. While these approaches work relatively well at smaller scales, they fail to scale to leadership-class computing systems because of the intense metadata loads generated they generate. Recent research identified significant improvements from using shared-file I/O instead of multifile I/O patterns on leadership-class systems [6]. In this paper, we propose integrating shared file support into the I/O forwarding layer commonly found on leadership-class computing systems. I/O forwarding middleware, such as the I/O Forwarding Scalability Layer (IOFSL) [9,2], bridges the compute and I/O subsystems of leadership-class computing systems. This middleware layer captures all file I/O requests generated by applications executing on compute nodes and forwards them to dedicated I/O nodes. These I/O nodes, a common hardware feature of leadership-class computing systems, execute the I/O requests on behalf of the application. The I/O forwarding layer on these system is best suited to provide and manage shared file pointers because it has access to all application I/O requests and can provide enhanced file I/O capabilities independent of the system and I/O software stack. By embedding this capability into the I/O forwarding layer, applications developers can utilize shared file pointers for a variety of file I/O APIs (MPI-IO, POSIX, and ZOIDFS), synchronization levels (collective and independent I/O), and computing systems (IBM Blue Gene and Cray XT systems). We are adding several features to IOFSL and ROMIO to enable portable MPI-IO shared file pointer access. In prior work, we extended the ZOIDFS API [2] to provide a distributed atomic append capability. Our current work extends and generalizes this capability to provide shared file pointers as defined by the MPI standard. First, we created a per file shared (key,v...

HyCache: A User-Level Caching Middleware for Distributed File Systems

“…FusionFS has already scaled to 1K nodes, and we aim to scale up FusionFS+HyCache to 10K nodes. We will also apply HyCache to Many-Task Computing (MTC) [31][32][33][34], which has specific emphasis on data-intensive computing [35] and cloud computing [36].…”

ZHT: A Light-Weight Reliable Persistent Dynamic Scalable Zero-Hop Distributed Hash Table

“…FusionFS is optimized for a subset of HPC and many-task computing (MTC) [12,59,62,63] workloads, and it is designed for extreme scales [61]. These workloads are often extremely data-intensive [56,58,60], and optimizing data locality [55] becomes critical to achieving good scalability and performance.…”

Modeling Many-Task Computing Workloads on a Petaflop IBM Blue Gene/P Supercomputer

“…We investigated the largest available trace of real MTC workloads, collected over a 17-month period comprising of 173M tasks [38] [39]. We filtered out the logs to isolate only the 160K-core IBM Blue Gene/P Intrepid supercomputer from Argonne National Laboratory, which netted about 34.8M tasks with the minimum runtime of 0 seconds, maximum runtime of 1469.62 seconds, average runtime of 95.20 seconds, and standard deviation of 188.08.…”