Comparative I/O workload characterization of two leadership class storage clusters

Abstract

The Oak Ridge Leadership Computing Facility (OLCF) is a leader in large-scale parallel file system development, design, deployment and continuous operation. For the last decade, the OLCF has designed and deployed two large center-wide parallel file systems. The first instantiation, Spider 1, served the Jaguar supercomputer and its predecessor, Spider 2, now serves the Titan supercomputer, among many other OLCF computational resources. The OLCF has been rigorously collecting file and storage system statistics from these Spider systems since their transition to production state. In this paper we present the collected I/O workload statistics from the Spider 2 system and compare it to the Spider 1 data. Our analysis show that the Spider 2 workload is more more write-heavy I/O compared to Spider 1 (75% vs. 60%, respectively). The data also show the OLCF storage policies such as periodic purges are effectively managing the capacity resource of Spider 2. Furthermore, due to improvements in tdm multipath and ib srp software, we are utilizing the Spider 2 system bandwidth and latency resources more effectively. The Spider 2 bandwidth usage statistics shows that our system is working within the design specifications. However, it is also evident that our scientific applications can be more effectively served by a burst buffer storage layer. All the data has been collected by monitoring tools developed for the Spider ecosystem. We believe the observed data set and insights will help us better design the next-generation Spider file and storage system. It will also be helpful to the larger community for building more effective large-scale file and storage systems.

Characterization of a Big Data Storage Workload in the Cloud

“…The read size is small compared other studies with 90% reads smaller than 1KB. In other studies 90% of reads are smaller than 1KB in web caches [2], 100KB in HPC [3], 1MB in HPC [13], 4MB in consumer cloud [20], 10MB in video delivery [25] and 15MB in HBase [14]. Figure 7 depicts that distributions of the two traces W1 and W2 are not too different.…”

“…We have collected from Databricks a combined Spark and S3 workload trace spanning over 6 months of operation, and designed a method to process and analyze this trace. Our method focuses HPC ✓ ✓ Abad 2012 [1] MapReduce + HDFS ✓ ✓ ✓ ✓ ✓ Chen 2012 [5] MapReduce + HDFS ✓ ✓ ✓ Atikoglu 2012 [2] Web Cache ✓ ✓ ✓ ✓ Liu 2013 [20] Consumer Cloud ✓ ✓ ✓ Harter 2014 [14] Messaging + HDFS ✓ ✓ ✓ Gunasekaran 2015 [13] HPC ✓ ✓ Summers 2016 [25] Video Delivery ✓ ✓…”

Performance Study of Non-volatile Memories on a High-End Supercomputer

“…The behavior of I/O depending on the transfer size is important in order to estimate I/O performance in scientific applications in HPC. According to the study [4] of the storage clusters Spider and Spider II at the Oak Ridge Leadership Facility (OLCF), about 50% of the requested transfer sizes are below 16KB and 50% between 512KB and 1MB in size. In figure 5 we can see that for 1MB transfer size, both local storage layers already have reached a region of maximal performance.…”

A Quantitative Approach to Architecting All-Flash Lustre File Systems

“…The flash cells within SSDs can be rewritten a finite number of times before they are no longer able to reliably store data, and as a result, SSDs are only warranted for a finite number of drive writes per day (DWPD) over their service life. Since HPC file systems have historically been subject to write-intensive workloads [10,22], the endurance requirements of SSDs in HPC environments have been a cause of concern. To date, most large-scale flash deployments in HPC have resorted to using extreme-endurance SSDs (5-10 DWPD for a fiveyear period) to ensure that the SSDs do not fail before the end of the overall system's service life [11,19].…”