Data Storage Systems of «hybrilit» Heterogeneous Computing Platform for Scientfic Research Carried Out in Jinr: Filesystems and Raids Performance Research

Abstract

"HybriLIT" Heterogeneous platform is a part of the Multifunctional Information and ComputingComplex (MICC) of the Laboratory of Information Technologies named after MG Meshcheryakov ofJINR, Dubna. Heterogeneous platform consists of Govorun supercomputer and HybriLIT educationand testing polygon. Data storage and processing system is one of the platform components. It isimplemented using distributed and parallel filesystems (NFS, EOS, Lustre). Platform performancedepends on many factors, including performance of storage and file systems.The best storage performance for wide variety of user jobs may be obtains with optimal filesystemparameters. The number of tests of local filesystems (EXT family and XFS) was carried out. Therewere empirically obtained an optimal parameters o data storage system at which the performance havebeen high results.The new methodology was developed for analyzing the obtained measurements of IOPS (input-outputoperations per second) and Latency (milliseconds) for results evaluations.Various filesystems were analyzed by the developed methodology. The conclusion was drawn aboutof optimal parameters of the investigated filesystems.

Providing High-Speed Data Access for Parallel Computing in the HPC Cluster

“…Lustre's parallel file system achieves its performance by automatically splitting data into chunks known as "stripes" and writing the stripes in a round-robin fashion across multiple storage objects. This process, called "striping", can significantly increase file I/O speed by eliminating single disk bottlenecks [7].…”