DCDB Wintermute: Enabling Online and Holistic Operational Data Analytics on HPC Systems

Netti, Alessio; Mueller, Micha; Guillen, Carla; Ott, Michael; Tafani, Daniele; Ozer, Gence; Schulz, Martin

doi:10.1145/3369583.3392674

Cited by 22 publications

(17 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this system does not consider rootcause analysis of the performance reduction. The authors of [5], [39] do not only provide the feature of real-time monitoring, but are also able to identify the performance issues and trouble-shoot the cause of the issues. In addition to them, [40] uses a type of artificial neural network called autoencoder for anomaly detection.…”

Section: Related Workmentioning

confidence: 99%

AutoDiagn: An Automated Real-Time Diagnosis Framework for Big Data Systems

Demirbaga

Wen

Noor

et al. 2022

IEEE Trans. Comput.

View full text Add to dashboard Cite

Big data processing systems, such as Hadoop and Spark, usually work in large-scale, highly-concurrent, and multi-tenant environments that can easily cause hardware and software malfunctions or failures, thereby leading to performance degradation. Several systems and methods exist to detect big data processing systems' performance degradation, perform root-cause analysis, and even overcome the issues causing such degradation. However, these solutions focus on specific problems such as stragglers and inefficient resource utilization. There is a lack of a generic and extensible framework to support the real-time diagnosis of big data systems. In this paper, we propose, develop and validate AutoDiagn. This generic and flexible framework provides holistic monitoring of a big data system while detecting performance degradation and enabling root-cause analysis. We present an implementation and evaluation of AutoDiagn that interacts with a Hadoop cluster deployed on a public cloud and tested with real-world benchmark applications. Experimental results show that AutoDiagn can offer a high accuracy root-cause analysis framework, at the same time as offering a small resource footprint, high throughput and low latency.

show abstract

Section: Related Workmentioning

confidence: 99%

AutoDiagn: An Automated Real-Time Diagnosis Framework for Big Data Systems

Demirbaga

Wen

Noor

et al. 2022

IEEE Trans. Comput.

View full text Add to dashboard Cite

show abstract

“…Based on an extensive survey of experimental techniques proposed in the literature [3, 13, 18, 20, 21, 23, 26, 30, 34, 35, 40-42, 45, 47] and in light of our long-term experiences at LRZ [36][37][38], we derive a generic formulation for ODA -namely, we identify the main functional steps composing it. These are summarized in Figure 1: the first step consists in Monitoring of system resources by collecting sensor or log data; this is followed by Monitoring Data Processing, transforming the raw monitoring data into a polished representation that can be comprehended by ODA techniques -aggregation and dimensionality reduction, for example, are two common approaches to solve this task.…”

Section: State Of the Art And System Designmentioning

confidence: 99%

“…A wide variety of software solutions falling under our definition of ODA are available in the literature, aiming to cover the different aspects of an HPC system's operation [37]. The vast majority of these techniques have never been employed in a production context, and thus their practical applicability is not proven: on the other hand, a small subset of tools are indeed known to have been used in production environments, but are either tailored for individual use cases or lack any report of long-term operation.…”

Section: State Of the Artmentioning

confidence: 99%

“…The single steps of ODA entail different operational requirements for working effectively in production environments. As these have been explored separately in previous work [36][37][38], here we focus on the overarching requirements of an ODA pipeline as a whole:…”

Section: Requirements and Toolsmentioning

confidence: 99%

“…• Wintermute [37] is a generic ODA framework designed to be integrated into any monitoring solution, with operator plugins supplying arbitrary analysis or control capabilities in a flexible manner. The currently available plugins allow for aggregation of data, computation of per-job metrics, output of sensors to arbitrary sinks, and machine learning tasks such as regression, classification and clustering.…”

Section: Requirements and Toolsmentioning

confidence: 99%

See 2 more Smart Citations

Operational Data Analytics in Practice: Experiences from Design to Deployment in Production HPC Environments

Netti¹,

Ott²,

Guillen³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

As HPC systems grow in complexity, efficient and manageable operation is increasingly critical. Many centers are thus starting to explore the use of Operational Data Analytics (ODA) techniques, which extract knowledge from massive amounts of monitoring data and use it for control and visualization purposes. As ODA is a multifaceted problem, much effort has gone into researching its separate aspects: however, accounts of production ODA experiences are still hard to come across.In this work we aim to bridge the gap between ODA research and production use by presenting our experiences with ODA in production, involving in particular the control of cooling infrastructures and visualization of job data on two HPC systems. We cover the entire development process, from design to deployment, highlighting our insights in an effort to drive the community forward. We rely on open-source tools, which make for a generic ODA framework suitable for most scenarios.

show abstract

Towards bespoke optimizations of energy efficiency in HPC environments

Tracey,

Elisseev,

Smyrnakis

et al. 2023

Applied AI Letters

View full text Add to dashboard Cite

We present bespoke energy efficiency optimizations in high performance computing (HPC) environments using holistic approach to data collection, analysis and proactive management of resources and workloads. Our solution has three major components: (i) platform for collecting, storing and processing data from multiple sources across hardware and software stacks, (ii) collections of regression machine learning (ML) algorithms for making workloads classifications and energy usage predictions, (iii) agent‐based decision‐making framework for delivering control decisions to middle‐ware and infrastructure thus supporting real time or near real energy efficiency optimizations. We will provide some concrete examples of using our proposed approach in HPC environment.

show abstract

DCDB Wintermute: Enabling Online and Holistic Operational Data Analytics on HPC Systems

Cited by 22 publications

References 34 publications

AutoDiagn: An Automated Real-Time Diagnosis Framework for Big Data Systems

AutoDiagn: An Automated Real-Time Diagnosis Framework for Big Data Systems

Operational Data Analytics in Practice: Experiences from Design to Deployment in Production HPC Environments

Towards bespoke optimizations of energy efficiency in HPC environments

Contact Info

Product

Resources

About