An Ephemeral Burst-Buffer File System for Scientific Applications

Wang, Teng; Mohror, Kathryn; Adam, Moody; Sato, Kento; Yu, Weikuan

doi:10.1109/sc.2016.68

Cited by 75 publications

(44 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…A burst buffer can also be located inside the main memory of a compute node or as a fast non-volatile storage device placed in the compute node (i.e., NVRAM devices, SSD devices, PCM devices, etc.). BurstFS [38] is an SSD-based distributed file system to be used as burst buffer for scientific applications. It buffers the application checkpoints to the local SSDs placed in the compute nodes, while asynchronously flushing them to the parallel file system.…”

Section: Nvfs-based Burst Buffer For Spark Over Lustrementioning

confidence: 99%

“…As a result, unlike Spark running over Lustre, the tasks do not have to write data over the network to the shared file system, rather, data is asynchronously flushed to Lustre. Besides, studies have already shown that, in terms of data access peak bandwidth, SSD is much faster than Lustre [20,22,38]. NVM being faster than SSD, has much higher performance than Lustre.…”

Section: Pagerankmentioning

confidence: 99%

See 1 more Smart Citation

High Performance Design for HDFS with Byte-Addressability of NVM and RDMA

Islam

Wasi-ur-Rahman

et al. 2016

Proceedings of the 2016 International Conference on Supercomputing

View full text Add to dashboard Cite

Non-Volatile Memory (NVM) offers byte-addressability with DRAM like performance along with persistence. Thus, NVMs provide the opportunity to build high-throughput storage systems for data-intensive applications. HDFS (Hadoop Distributed File System) is the primary storage engine for MapReduce, Spark, and HBase. Even though HDFS was initially designed for commodity hardware, it is increasingly being used on HPC (High Performance Computing) clusters. The outstanding performance requirements of HPC systems make the I/O bottlenecks of HDFS a critical issue to rethink its storage architecture over NVMs. In this paper, we present a novel design for HDFS to leverage the byteaddressability of NVM for RDMA (Remote Direct Memory Access)-based communication. We analyze the performance potential of using NVM for HDFS and re-design HDFS I/O with memory semantics to exploit the byte-addressability fully. We call this design NVFS (NVM-and RDMA-aware HDFS). We also present cost-effective acceleration techniques for HBase and Spark to utilize the NVM-based design of HDFS by storing only the HBase Write Ahead Logs and Spark job outputs to NVM, respectively. We also propose enhancements to use the NVFS design as a burst buffer for running Spark jobs on top of parallel file systems like Lustre. Performance evaluations show that our design can improve the write and read throughputs of HDFS by up to 4x and 2x, respectively. The execution times of data generation benchmarks are reduced by up to 45%. The proposed design also reduces the overall execution time of the SWIM workload by up to 18% over HDFS with a maximum benefit of 37% for job-38. For Spark TeraSort, our proposed scheme yields a performance gain of up to 11%. The performances of HBase insert, update, and read operations are improved by 21%, 16%, and 26%, respectively. Our NVM-based burst buffer can improve the I/O performance of Spark PageRank by up *

show abstract

Section: Nvfs-based Burst Buffer For Spark Over Lustrementioning

confidence: 99%

Section: Pagerankmentioning

confidence: 99%

High Performance Design for HDFS with Byte-Addressability of NVM and RDMA

Islam

Wasi-ur-Rahman

et al. 2016

Proceedings of the 2016 International Conference on Supercomputing

View full text Add to dashboard Cite

show abstract

“…ere exist di erent architectures for implementing the burst bu er. Two typical architectures are I/O server-oriented architecture [29] [16] and computing node-oriented architecture [31] [41]. In the I/O server-oriented architecture, the burst bu er system is deployed as an I/O server, which accepts the I/O requests from clients (e.g., computing nodes), while in the computing node-oriented architecture, the burst bu er mechanism is implemented inside the local computing node, handling the I/O requests issued by the local node.…”

Section: Overview Of Ssdup+mentioning

confidence: 99%

“…In addition, Hystor is designed only for stand-alone and embedded kernels, not for parallel environment. BurstFS [31] carefully analyzes the data characteristics of scienti c applications to access the data e ciently. But its goal is to pursue the optimal performance, without considering the e ciency and cost-e ectiveness of SSD.…”

Section: Storage Systems With Burst Bu Ermentioning

confidence: 99%

Optimizing the SSD Burst Buffer by Traffic Detection

Shi

Liu

et al. 2020

ACM Trans. Archit. Code Optim.

View full text Add to dashboard Cite

Currently, HPC storage systems still use hard disk drive (HDD) as their dominant storage device. Solid state drive (SSD) is widely deployed as the bu er to HDDs. Burst bu er has also been proposed to manage the SSD bu ering of bursty write requests. Although burst bu er can improve I/O performance in many cases, we nd that it has some limitations such as requiring large SSD capacity and harmonious overlapping between computation phase and data ushing phase.In this paper, we propose a scheme, called SSDUP+ 1 . SSDUP+ aims to improve the burst bu er by addressing the above limitations. First, in order to reduce the demand for the SSD capacity, we develop a novel method to detect and quantify the data randomness in the write tra c. Further, an adaptive algorithm is proposed to classify the random writes dynamically. By doing so, much less SSD capacity is required to achieve the similar performance as other burst bu er schemes. Next, in order to overcome the di culty of perfectly overlapping the computation phase and the ushing phase, we propose a pipeline mechanism for the SSD bu er, in which data bu ering and ushing are performed in pipeline. In addition, in order to improve the I/O throughput, we adopt a tra c-aware ushing strategy to reduce the I/O interference in HDD. Finally, in order to further improve the performance of bu ering random writes in SSD, SSDUP+ transforms the random writes to sequential writes in SSD by storing the data with a log structure. Further, SSDUP+ uses the AVL tree structure to store the sequence information of the data.We have implemented a prototype of SSDUP+ based on OrangeFS and conducted extensive experiments. e experimental results show that our proposed SSDUP+ can save an average of 50% SSD space, while delivering almost the same performance as other common burst bu er schemes. In addition, SSDUP+ can save about 20% SSD space compared with the previous version of this work, SSDUP, while achieving 20%-30% higher I/O throughput than SSDUP.

show abstract

“…The decoupling technique has shown its effectiveness in developing runtimes on HPC platforms, such as burst-buffer file systems and load balancing framework [22], [13]. However, decoupling general operations at the application-level has not been explored in depth.…”

Section: Ipic3dmentioning

confidence: 99%

Preparing HPC Applications for the Exascale Era: A Decoupling Strategy

Gioiosa

Kestor

Laure

2017

2017 46th International Conference on Parallel Processing (ICPP)

View full text Add to dashboard Cite

Production-quality parallel applications are often a mixture of diverse operations, such as computation-and communication-intensive, regular and irregular, tightly coupled and loosely linked operations. In conventional construction of parallel applications, each process performs all the operations, which might result inefficient and seriously limit scalability, especially at large scale. We propose a decoupling strategy to improve the scalability of applications running on large-scale systems. Our strategy separates application operations onto groups of processes and enables a dataflow processing paradigm among the groups. This mechanism is effective in reducing the impact of load imbalance and increases the parallel efficiency by pipelining multiple operations. We provide a proof-of-concept implementation using MPI, the de-facto programming system on current supercomputers. We demonstrate the effectiveness of this strategy by decoupling the reduce, particle communication, halo exchange and I/O operations in a set of scientific and dataanalytics applications. A performance evaluation on 8,192 processes of a Cray XC40 supercomputer shows that the proposed approach can achieve up to 4× performance improvement.

show abstract

An Ephemeral Burst-Buffer File System for Scientific Applications

Cited by 75 publications

References 20 publications

High Performance Design for HDFS with Byte-Addressability of NVM and RDMA

High Performance Design for HDFS with Byte-Addressability of NVM and RDMA

Optimizing the SSD Burst Buffer by Traffic Detection

Preparing HPC Applications for the Exascale Era: A Decoupling Strategy

Contact Info

Product

Resources

About