A Multi-faceted Approach to Job Placement for Improved Performance on Extreme-Scale Systems

Zimmer, Christopher; Gupta, Saurabh; Atchley, Scott; Vazhkudai, Sudharshan S.; Albing, Carl

doi:10.1109/sc.2016.86

Cited by 11 publications

(2 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Replicated execution can decrease the perceived failure rate of the job. But, some job schedulers such Moab and scheduling techniques pack jobs in space to improve locality [41]. If replicated job is submitted under the same batch script, it is likely to experience higher failure rate due to neighborhood recurrence property.…”

Section: Mtbf×num Of Nodes In the System Max Number Of Nodes Across Amentioning

confidence: 99%

Failures in large scale systems

Gupta

Patel

Engelmann

et al. 2017

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

105

View full text Add to dashboard Cite

Resilience is one of the key challenges in maintaining high eiciency of future extreme scale supercomputers. Researchers and system practitioners rely on ield-data studies to understand reliability characteristics and plan for future HPC systems. In this work, we compare and contrast the reliability characteristics of multiple largescale HPC production systems. Our study covers more than one billion compute node hours across ive diferent systems over a period of 8 years. We conirm previous indings which continue to be valid, discover new indings, and discuss their implications.

show abstract

Section: Mtbf×num Of Nodes In the System Max Number Of Nodes Across Amentioning

confidence: 99%

Failures in large scale systems

Gupta

Patel

Engelmann

et al. 2017

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

105

View full text Add to dashboard Cite

show abstract

“…Scheduling data (Section 4.2.3) from GUIDE has impacted operations measurably [29]. Using insights from data mining and visualization, it became apparent that the resource manager on Titan can be tweaked to better suit our workload.…”

Section: Scheduling Use Casesmentioning

confidence: 99%

Guide

Vazhkudai

Miller

Tiwari

et al. 2017

Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Self Cite

View full text Add to dashboard Cite

In this paper, we describe the GUIDE framework used to collect, federate, and analyze log data from the Oak Ridge Leadership Computing Facility (OLCF), and how we use that data to derive insights into facility operations. We collect system logs and extract monitoring data at every level of the various OLCF subsystems, and have developed a suite of pre-processing tools to make the raw data consumable. The cleansed logs are then ingested and federated into a central, scalable data warehouse, Splunk, that offers storage, indexing, querying, and visualization capabilities. We have further developed and deployed a set of tools to analyze these multiple disparate log streams in concert and derive operational insights. We describe our experience from developing and deploying the GUIDE infrastructure, and deriving valuable insights on the various subsystems, based on two years of operations in the production OLCF environment.

show abstract