DAOS and Friends: A Proposal for an Exascale Storage System

Lofstead, Jay; Jimenez, Ivo; Maltzahn, Carlos; Koziol, Quincey; Bent, John; Barton, Eric

doi:10.1109/sc.2016.49

Cited by 61 publications

(29 citation statements)

References 26 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…This generates an enormous amount of write traffic to storage systems (Rajachandrasekar et al, 2013). Much work (Ali et al, 2009;Dongarra, 2010;Shalf et al, 2010;Bent et al, 2012;Lofstead et al, 2016) has shown that current HDD-based storage systems have been stretched to their limits in handling the tremendous amount of I/O.…”

Section: Introductionmentioning

confidence: 99%

“…The US Department of Energy and seven leading US national laboratories initiated a project named 'fast forward storage and I/O (FFSIO)' (Lofstead et al, 2016) to develop an I/O stack suitable for extreme-scale systems. It suggests processing the enormous amount of concurrent I/O in a staging area closer to compute nodes, so that applications can return to computation as soon as possible.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

ONFS: a hierarchical hybrid file system based on memory, SSD, and HDD for high performance computers

Liu

Yu³

et al. 2017

Frontiers Inf Technol Electronic Eng

View full text Add to dashboard Cite

With supercomputers developing towards exascale, the number of compute cores increases dramatically, making more complex and larger-scale applications possible. The input/output (I/O) requirements of large-scale applications, workflow applications, and their checkpointing include substantial bandwidth and an extremely low latency, posing a serious challenge to high performance computing (HPC) storage systems. Current hard disk drive (HDD) based underlying storage systems are becoming more and more incompetent to meet the requirements of next-generation exascale supercomputers. To rise to the challenge, we propose a hierarchical hybrid storage system, on-line and near-line file system (ONFS). It leverages dynamic random access memory (DRAM) and solid state drive (SSD) in compute nodes, and HDD in storage servers to build a three-level storage system in a unified namespace. It supports portable operating system interface (POSIX) semantics, and provides high bandwidth, low latency, and huge storage capacity. In this paper, we present the technical details on distributed metadata management, the strategy of memory borrow and return, data consistency, parallel access control, and mechanisms guiding downward and upward migration in ONFS. We implement an ONFS prototype on the TH-1A supercomputer, and conduct experiments to test its I/O performance and scalability. The results show that the bandwidths of single-thread and multi-thread 'read'/'write' are 6-fold and 5-fold better than HDD-based Lustre, respectively. The I/O bandwidth of data-intensive applications in ONFS can be 6.35 times that in Lustre.

show abstract

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

ONFS: a hierarchical hybrid file system based on memory, SSD, and HDD for high performance computers

Liu

Yu³

et al. 2017

Frontiers Inf Technol Electronic Eng

View full text Add to dashboard Cite

show abstract

“…We recognise the performance and functionality benefits that exploiting new storage technologies can bring to applications. We are therefore investigating the use of object stores, such as DAOS [21] and dataClay [22] and are porting them to the hardware architecture we are proposing, i.e. systems with distributed B-APM as the main storage hardware.…”

Section: Object Storementioning

confidence: 99%

An Architecture for High Performance Computing and Data Systems Using Byte-Addressable Persistent Memory

Jackson

Weiland

Parsons

et al. 2019

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Non-volatile, byte addressable, memory technology with performance close to main memory promises to revolutionise computing systems in the near future. Such memory technology provides the potential for extremely large memory regions (i.e. > 3TB per server), very high performance I/O, and new ways of storing and sharing data for applications and workflows. This paper outlines an architecture that has been designed to exploit such memory for High Performance Computing and High Performance Data Analytics systems, along with descriptions of how applications could benefit from such hardware.

show abstract

“…The Fast Forward Storage and IO (FFSIO) project aims at providing an exascale storage system which is capable of dealing with the requirements of HPC applications, as well as big data type workloads. It aims at introducing a new I/O stack and supporting more complex basic data types like containers and key-arrays [60]. Its functionality ranges from a general I/O interface at the top over an I/O forwarding and an I/O dispatcher layer to the Distributed Application Object Store (DAOS) layer, which offers a persistent storage interface and translates the object model visible to the user to the demands of the underlying infrastructure.…”

Section: Storage and File Systemsmentioning

confidence: 99%

Survey of Storage Systems for High-Performance Computing

Lüttgau

Kuhn

Duwe

et al. 2018

JSFI

View full text Add to dashboard Cite

In current supercomputers, storage is typically provided by parallel distributed file systems for hot data and tape archives for cold data. These file systems are often compatible with local file systems due to their use of the POSIX interface and semantics, which eases development and debugging because applications can easily run both on workstations and supercomputers. There is a wide variety of file systems to choose from, each tuned for different use cases and implementing different optimizations. However, the overall application performance is often held back by I/O bottlenecks due to insufficient performance of file systems or I/O libraries for highly parallel workloads. Performance problems are dealt with using novel storage hardware technologies as well as alternative I/O semantics and interfaces. These approaches have to be integrated into the storage stack seamlessly to make them convenient to use. Upcoming storage systems abandon the traditional POSIX interface and semantics in favor of alternative concepts such as object and key-value storage; moreover, they heavily rely on technologies such as NVM and burst buffers to improve performance. Additional tiers of storage hardware will increase the importance of hierarchical storage management. Many of these changes will be disruptive and require application developers to rethink their approaches to data management and I/O. A thorough understanding of today's storage infrastructures, including their strengths and weaknesses, is crucially important for designing and implementing scalable storage systems suitable for demands of exascale computing.

show abstract

DAOS and Friends: A Proposal for an Exascale Storage System

Cited by 61 publications

References 26 publications

ONFS: a hierarchical hybrid file system based on memory, SSD, and HDD for high performance computers

ONFS: a hierarchical hybrid file system based on memory, SSD, and HDD for high performance computers

An Architecture for High Performance Computing and Data Systems Using Byte-Addressable Persistent Memory

Survey of Storage Systems for High-Performance Computing

Contact Info

Product

Resources

About