Proctor: A Semi-Supervised Performance Anomaly Diagnosis Framework for Production HPC Systems

Aksar, Burak; Zhang, Yijia; Ateş, Emre; Schwaller, Benjamin; Aaziz, Omar; Leung, Vitus J.; Brandt, James M.; Egele, Manuel; Coskun, Ayse K.

doi:10.1007/978-3-030-78713-4_11

Cited by 9 publications

(5 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…As this is the first attempt at employing FL for this task, we decided to focus on a specific approach for anomaly detection in HPC, namely the semi-supervised method, as it has demonstrated very good results when applied to real HPC while not requiring fully annotated data (which is not always available in the HPC context); in future works, we plan to explore the impact of FL to other underlying ML models for anomaly detection. We start with a key assumption (the same described in previous semi-supervised approaches for HPC anomaly detection, see [5], [10]), that is we suppose that the training data will contain only examples corresponding to normal operating condition; this is a safe assumption as it is typically possible to identify relatively long healthy periods right after the installation of a new supercomputer before the HW components start to degrade. To allow for a fair comparison (FL usage against no FL) we opted to use as baseline ML model the same semisupervised deep neural network (DNN) employed in [10] 3 , namely an autoencoder network.…”

Section: Methodsmentioning

confidence: 99%

See 1 more Smart Citation

A Federated Learning Approach for Anomaly Detection in High Performance Computing

Farooq,

Borghesi

2023

2023 IEEE 35th International Conference on Tools With Artificial Intelligence (ICTAI)

View full text Add to dashboard Cite

High Performance Computing (HPC) systems are complex machines that need to be operated at their maximum potential to recoup their investment cost and to mitigate their environmental impact. Anomalous conditions hindering the correct usage of the supercomputing nodes are a significant problem. Hence, the development of automated anomaly detection techniques remains a vital area of research. Machine Learning (ML) models demonstrated to be good at detecting anomalies on individual nodes. However, the potential of combining data from multiple computing nodes and associated ML models has not been explored yet. Federated Learning (FL) can address this shortcoming, by allowing individual models to learn from each other. This paper applies FL to improve the performance of anomaly detection models for HPC systems. The approach has been validated on data from an actual supercomputer, obtaining an improvement in the average f-score from 0.31 to 0.84. We also show how FL can significantly shorten the data collection period needed to create a training set. While ML models need, on average, 4.5 months of training data, FL reduces the training set size to 1.2 weeks -a 15x reduction.

show abstract

Section: Methodsmentioning

confidence: 99%

“…Most data-driven approaches drew inspiration from Machine Learning (ML) and Deep Learning (DL) domain [5], including supervised approaches that assume labeled training data [6] and unsupervised ones [7]. Semi-supervised models have been demonstrated to be very good at merging the strengths from both areas' weaknesses [8].…”

Section: Introductionmentioning

confidence: 99%

A Federated Learning Approach for Anomaly Detection in High Performance Computing

Farooq,

Borghesi

2023

2023 IEEE 35th International Conference on Tools With Artificial Intelligence (ICTAI)

View full text Add to dashboard Cite

show abstract

“…The XE were provided in the Digital Imaging and Communications in Medicine (DICOM) format. All XE were transformed from DICOM-format to NPY, which is readable for CNN architectures [21]. After the transformation, it was important to ensure that all the images were appropriately resized.…”

Section: Dataset Preparationmentioning

confidence: 99%

Quality Assessment Assistance of Lateral Knee X-rays: A Hybrid Convolutional Neural Network Approach

et al. 2023

View full text Add to dashboard Cite

A common issue with X-ray examinations (XE) is the erroneous quality classification of the XE, implying that the process needs to be repeated, thus delaying the diagnostic assessment of the XE and increasing the amount of radiation the patient receives. The authors propose a system for automatic quality classification of XE based on convolutional neural networks (CNN) that would simplify this process and significantly decrease erroneous quality classification. The data used for CNN training consist of 4000 knee images obtained via radiography procedure (KXE) in total, with 2000 KXE labeled as acceptable and 2000 as unacceptable. Additionally, half of the KXE belonging to each label are right knees and left knees. Due to the sensitivity to image orientation of some CNNs, three approaches are discussed: (1) Left-right-knee (LRK) classifies XE based just on their label, without taking into consideration their orientation; (2) Orientation discriminator (OD) for the left knee (LK) and right knee (RK) analyses images based on their orientation and inserts them into two separate models regarding orientation; (3) Orientation discriminator combined with knee XRs flipped to the left or right (OD-LFK)/OD-RFK trains the models with all images being horizontally flipped to the same orientation and uses the aforementioned OD to determine whether the image needs to be flipped or not. All the approaches are tested with five CNNs (AlexNet, ResNet50, ResNet101, ResNet152, and Xception), with grid search and k-fold cross-validation. The best results are achieved using the OD-RFK hybrid approach with the Xception network architecture as the classifier and ResNet152 as the OD, with an average AUC of 0.97 (±0.01).

show abstract

“…All mentioned approaches do not take into account temporal dependencies of data (models are not trained on time series but on tabular data containing no temporal information). System monitoring data approach [47] is the first to take into account temporal dependencies in data by calculating statistical features on temporal dimension (aggregation, sliding window statistics, lag features). Most approaches that deal with time series anomaly detection do so on system log data.…”

Section: Related Workmentioning

confidence: 99%

Ruad: Unsupervised Anomaly Detection in Hpc Systems

Molan¹,

Borghesi

Cesarini³

et al. 2022

SSRN Journal

View full text Add to dashboard Cite

Proctor: A Semi-Supervised Performance Anomaly Diagnosis Framework for Production HPC Systems

Cited by 9 publications

References 34 publications

A Federated Learning Approach for Anomaly Detection in High Performance Computing

A Federated Learning Approach for Anomaly Detection in High Performance Computing

Quality Assessment Assistance of Lateral Knee X-rays: A Hybrid Convolutional Neural Network Approach

Ruad: Unsupervised Anomaly Detection in Hpc Systems

Contact Info

Product

Resources

About