Usage behavior of a large-scale scientific archive

Adams, Ian F.; Madden, Brian A.; Frank, Joel C.; Storer, Mark W.; Miller, Ethan L.; Harano, Gene

doi:10.1109/sc.2012.110

Cited by 9 publications

(4 citation statements)

References 14 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The definitions for cold storage and archival storage suggest a data set that may be accessed less frequently than hot or high intensity storage systems; however, there is no exact definition for an archival storage system. Previous studies have analyzed storage system behavior under different workloads, including both archival and high intensity workloads [5], [8], [9]. The challenge of rigorously characterizing storage system workloads does not outweigh the importance of using workload characteristics to understand how best to model and design storage systems [10].…”

Section: Archive Parametersmentioning

confidence: 99%

Using Simulation to Design Scalable and Cost-Efficient Archival Storage Systems

Byron

Long

Miller

2018

2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS

Self Cite

View full text Add to dashboard Cite

The need for reliable and cost-effective data storage grows as digital information becomes increasingly ubiquitous. Archival systems must store valuable data for years while adapting to changing user needs, capacity, and performance requirements. Storage devices differ in terms of performance, capacity, reliability, acquisition cost, power consumption, and the rates at which their features change over time. As a result, choosing the best storage technology to use for an archive has become increasingly challenging with the proliferation of new technologies alongside existing ones.We have designed a simulator that models the capacity, performance, acquisition cost, and power cost of an archival system using the characteristics of the drives and media that comprise it. We simulate and compare four storage technologies that exhibit different cost and performance characteristics: tape, optical disc, hard disk, and NAND flash SSD. We evaluate the total cost of ownership for each storage technology within an archival system, and we explore the effect that prospective technological advancements and growth rates over time may have on the relative cost and viability of each storage technology for archival systems. We show that the lifecycle and upgrade cost of drives are significant cost factors for removable media archives. We observe that increasing performance requires adding more drives to an archival system, and the cost of each drive dominates the cost to increase performance. We compare trends in storage technologies to suggest developments that could minimize the long-term total cost of ownership for archival systems. We show that hard disks and flash could become cost-competitive with tape-based archives by adopting new designs to minimize infrastructure and electricity costs.

show abstract

Section: Archive Parametersmentioning

confidence: 99%

Using Simulation to Design Scalable and Cost-Efficient Archival Storage Systems

Byron

Long

Miller

2018

2018 IEEE 26th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS

Self Cite

View full text Add to dashboard Cite

show abstract

“…Because of this difference in log/trace format, system analysis is usually done per system. Such analysis can produce useful results on the behavior of a particular system, but sheds no light on how it compares to other similar systems, HPC or otherwise [6][7][8][9]. Additionally, programmer effort is wasted analyzing each system when the generated analytics are the same or similar for each system.…”

Section: Analysis Of Datamentioning

confidence: 99%

“…Additional tools of note are the Integrated Performance Monitoring for HPC 6 , IOSIG [19], RIOT [20], ScalaIOTrace [5], and Linux blktrace.…”

Section: Additional User-level Toolsmentioning

confidence: 99%

Tools for Analyzing Parallel I/O

Kunkel

Betke

Bryson

et al. 2018

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Parallel application I/O performance often does not meet user expectations. Additionally, slight access pattern modifications may lead to significant changes in performance due to complex interactions between hardware and software. These issues call for sophisticated tools to capture, analyze, understand, and tune application I/O. In this paper, we highlight advances in monitoring tools to help address these issues. We also describe best practices, identify issues in measurement and analysis, and provide practical approaches to translate parallel I/O analysis into actionable outcomes for users, facility operators, and researchers.

show abstract

“…Identifying working sets accurately and reliably can greatly improve both the performance and the efficiency of storage systems [3]. Additionally, understanding workload characteristics is essential for optimal storage management and provisioning [1].…”

Section: Introductionmentioning

confidence: 99%

Single-Snapshot File System Analysis

Wildani

Adams

Miller

2013

2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems

Self Cite

View full text Add to dashboard Cite

Abstract-Metadata snapshots are a common method for gaining insight into filesystems due to their small size and relative ease of acquisition. Since they are static, most researchers have used them for relatively simple analyses such as file size distributions and age of files. We hypothesize that it is possible to gain much richer insights into file system and user behavior by clustering features in metadata snapshots and comparing the entropy within clusters to the entropy within natural partitions such as directory hierarchies. We discuss several different methods for gaining deeper insights into metadata snapshots, and show a small proof of concept using data from Los Alamos National Laboratories. In our initial work, we see evidence that it is possible to identify user locality information, traditionally the purview of dynamic traces, using a single static snapshot.

show abstract

Usage behavior of a large-scale scientific archive

Cited by 9 publications

References 14 publications

Using Simulation to Design Scalable and Cost-Efficient Archival Storage Systems

Using Simulation to Design Scalable and Cost-Efficient Archival Storage Systems

Tools for Analyzing Parallel I/O

Single-Snapshot File System Analysis

Contact Info

Product

Resources

About