IO-Sets: Simple and Efficient Approaches for I/O Bandwidth Management

Abstract

One of the main performance issues faced by highperformance computing platforms is the congestion caused by concurrent I/O from applications. When this happens, the platform's overall performance and utilization are harmed. From the extensive work in this field, I/O scheduling is the essential solution to this problem. The main drawback of current techniques is the amount of information needed about applications, which compromises their applicability. In this paper, we propose a novel method for I/O management, IO-SETS. We present its potential through a scheduling heuristic called SET-10, which is simple and requires only minimal information. Our extensive experimental campaign shows the importance of IO-SETS and the robustness of SET-10 under various workloads. In particular in most of the simulated scenarios we improve the I/O slowdown over fairshare by 50%, which corresponds in our scenarios to a platform utilization gain of 2.5%. In the practical scenarios that we did, the utilization gain varies between 10 and 30%. We also provide insights on using our proposal in practice.

Revisiting I/O bandwidth-sharing strategies for HPC applications

“…In contrast, our approach does not assume any a priori knowledge of the concurrent applications. [27] This recent work can be viewed as an interesting extension of the work in [2] for periodic applications. Each application consists of several iterations, which as above are work phases followed by I/O operations.…”

“…Instead, for each application, they determine the value of ω, which is the average length of an iteration so far. In [27], CPU lengths and I/O volumes are sampled from some probability distributions (that differ for each application), which enables to compute ω with the expectations of these distributions, but one could envision to acquire the value of ω on the fly, as the application progresses. Then, the applications are partitioned into I/O-sets: two applications belong to the same set if they have the same value for log 10 ω .…”

“…Several researchers have already identified and addressed this problem (see [7,12,25,2,28,27] among others). Performance degradation due to I/O is already significant for current state-of-the-art platforms and is expected to worsen due to the faster increase in processing speed than in I/O bandwidth [22].…”

Revisiting I/O Bandwidth-Sharing Strategies for Hpc Applications

“…We start with some simple greedy strategies, some old and some new, in Section 4.1. Then in Section 4.2, we detail the recent Set-10 strategy proposed in [4]. Finally, in Section 4.3, we sketch an elaborate strategy whose aim is to compute the best horizon for maximizing the minimum yield.…”

“…bi , and b i = min (p i b, B ). Note that we neglect the initial I/O operation v (0) i to match the specification of [4]. Then, A i is assigned to I/O-set S n , where n = log 10 ω i , and x denotes the nearest integer to x.…”