Automatic I/O scheduling algorithm selection for parallel file systems

Boito, Francieli Zanon; Kassick, Rodrigo; Navaux, Philippe O. A.; Denneulin, Yves

doi:10.1002/cpe.3606

Cited by 21 publications

(18 citation statements)

References 22 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…For that, both postmortem and runtime approaches are popular. After the execution, information is often obtained from traces and applied to future executions of the same applications [5], [7], targeting repeated patterns with similar characteristics.…”

Section: Related Workmentioning

confidence: 99%

“…In general, optimization techniques typically provide improvements for particular system configurations and access patterns, but not for all of them. Furthermore, they often rely on the right selection of parameters, as demonstrated for request scheduling at different levels [5], [6]. Therefore, achieving the best results proposed by an optimization technique often relies on correctly applying them to the proper workload, and configuring it accordingly.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Detecting I/O Access Patterns of HPC Workloads at Runtime

Bez

Boito

Nou

et al. 2019

2019 31st International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)

Self Cite

View full text Add to dashboard Cite

In this paper, we seek to guide optimization and tuning strategies by identifying the application's I/O access pattern. We evaluate three machine learning techniques to automatically detect the I/O access pattern of HPC applications at runtime: decision trees, random forests, and neural networks. We focus on the detection using metrics from file-level accesses as seen by the clients, I/O nodes, and parallel file system servers. We evaluated these detection strategies in a case study in which the accurate detection of the current access pattern is fundamental to adjust a parameter of an I/O scheduling algorithm. We demonstrate that such approaches correctly classify the access pattern, regarding file layout and spatiality of accesses-into the most common ones used by the community and by I/O benchmarking tools to test new I/O optimization-with up to 99% precision. Furthermore, when applied to our study case, it guides a tuning mechanism to achieve 99% of the performance of an Oracle solution.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Detecting I/O Access Patterns of HPC Workloads at Runtime

Bez

Boito

Nou

et al. 2019

2019 31st International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD)

Self Cite

View full text Add to dashboard Cite

show abstract

“…To achieve such a goal, their algorithm relies on past I/O patterns of the applications to either maximize the global system utilization, or minimize the maximum slowdown induced by sharing bandwidth. Deeper in the I/O stack, at the I/O node level, the I/O ows can be reorganized to better match the characteristics of the storage devices [8].…”

Section: Related Workmentioning

confidence: 99%

A Methodology for Handling Data Movements by Anticipation: Position Paper

Bleuse

Lucarelli

Trystram

2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

The enhanced capabilities of large scale parallel and distributed platforms produce a continuously increasing amount of data which have to be stored, exchanged and used by various tasks allocated on dierent nodes of the system. The management of such a huge communication demand is crucial for reaching the best possible performance of the system. Meanwhile, we have to deal with more interferences as the trend is to use a single all-purpose interconnection network whatever the interconnect (tree-based hierarchies or topology-based heterarchies). There are two dierent types of communications, namely, the ows induced by data exchanges during the computations, and the ows related to Input/Output operations. We propose in this paper a general model for interference-aware scheduling, where explicit communications are replaced by external topological constraints. Specically, the interferences of both communication types are reduced by adding geometric constraints on the allocation of tasks into machines. The proposed constraints reduce implicitly the data movements by restricting the set of possible allocations for each task. This methodology has been proved to be ecient in a recent study for a restricted interconnection network (a line/ring of processors which is an intermediate between a tree and higher dimensions grids/torus). The obtained results illustrated well the diculty of the problem even on simple topologies, but also provided a pragmatic greedy solution, which was assessed to be ecient by simulations. We are currently extending this solution for more complex topologies. This work is a position paper which describes the methodology, it does not focus on the solving part.

show abstract

“…In , Application‐Guided I/O Scheduling for parallel file systems (AGIOS), a tool to improve the parallel input‐output (I/O) performance, is proposed. AGIOS is sensitive to storage device type.…”

Section: Papers In This Issuementioning

confidence: 99%