Online Anomaly Detection in HPC Systems

Borghesi, Andrea; Libri, Antonio; Benini, Luca; Bartolini, Andrea

doi:10.1109/aicas.2019.8771527

Cited by 35 publications

(36 citation statements)

References 19 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this direction, different methods were proposed to improve the technique, not only for MD, but also more in general for AD. In the context of DCs, works in [28]- [30], [32] showed how to use performance counters to detect anomalies, while [33] showed how to use them for detecting covert cryptocurrency mining. However, very recent works in [34], [35] bring into question the robustness of this technique for security, carrying out a study with a big dataset with more than ninety malware and reporting poor results in accuracy detection.…”

Section: Related Workmentioning

confidence: 99%

“…For the ML inference phase, we use an AE, which is a particular kind of Neural Network suitable for Anomaly Detection [30], [42]. As described in Section II, the idea is to train a model on "healthy" activity of the DC (i.e., with no malware involved), and try to identify possible anomalies in these signatures when a malware is running in background.…”

Section: B Edge Malware Detection Inferencementioning

confidence: 99%

See 1 more Smart Citation

pAElla: Edge AI-Based Real-Time Malware Detection in Data Centers

Libri

Bartolini

Benini

2020

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

The increasing use of Internet-of-Things (IoT) devices for monitoring a wide spectrum of applications, along with the challenges of "big data" streaming support they often require for data analysis, is nowadays pushing for an increased attention to the emerging edge computing paradigm. In particular, smart approaches to manage and analyze data directly on the network edge, are more and more investigated, and Artificial Intelligence (AI) powered edge computing is envisaged to be a promising direction. In this paper, we focus on Data Centers (DCs) and Supercomputers (SCs), where a new generation of high-resolution monitoring systems is being deployed, opening new opportunities for analysis like anomaly detection and security, but introducing new challenges for handling the vast amount of data it produces. In detail, we report on a novel lightweight and scalable approach to increase the security of DCs / SCs, that involves AI-powered edge computing on high-resolution power consumption. The method -called pAElla -targets real-time Malware Detection (MD), it runs on an out-of-band IoT-based monitoring system for DCs / SCs, and involves Power Spectral Density of power measurements, along with AutoEncoders. Results are promising, with an F1-score close to 1, and a False Alarm and Malware Miss rate close to 0%. We compare our method with State-of-the-Art MD techniques and show that, in the context of DCs / SCs, pAElla can cover a wider range of malware, significantly outperforming SoA approaches in terms of accuracy. Moreover, we propose a methodology for online training suitable for DCs / SCs in production, and release open dataset and code.

show abstract

Section: Related Workmentioning

confidence: 99%

Section: B Edge Malware Detection Inferencementioning

confidence: 99%

pAElla: Edge AI-Based Real-Time Malware Detection in Data Centers

Libri

Bartolini

Benini

2020

IEEE Internet Things J.

Self Cite

View full text Add to dashboard Cite

show abstract

“…Processing streaming data generated from HPC systems is challenging as a result of the large volumes and generating speed. An online supervised learning method is proposed in [49] to operate with live streamed data; Another online anomaly detection method using autoencoder is provided in [50]. To better support streaming logs analysis, a visual analytic framework is proposed in [51], which consists of data management, analysis and interactive visualization.…”

Section: Machine Learning and Hpc Securitymentioning

confidence: 99%

Security of HPC Systems: From a Log-analyzing Perspective

Luo¹,

Qu²,

Zeng³

et al. 2019

ICST Transactions on Security and Safety

View full text Add to dashboard Cite

High Performance Computing (HPC) systems mainly focused on how to improve performances of the computing. It has competitive processing capacity both in terms of calculation speed and available memory. HPC infrastructures are valuable computing resources that need to be carefully guarded and avoid being maliciously used. Thus, vulnerabilities are quintessential issues in HPC systems due to most of jobs and resources run or stored usually are sensitive and high-profit information. In this survey, we comprehensively review securities of HPC systems from a log-analyzing perspective, including well-known attacks and widely used defenses, especially intruder detection methods. We found that log files are used for the security purposes much less than what we expected. How to use all the available log files comprehensively and employ state-ofthe-art intrusion techniques to improve the robustness of HPC systems still lies for future research.

show abstract

“…With the fast growing of their size and complexity, modern information technology systems contain numerous sources of potential faults and vulnerabilities [3,6,8]. The failure of their services can have significant consequences, ranging from degraded user's experience [11] to important financial losses [15].…”

Section: Introductionmentioning

confidence: 99%

“…For instance, a late appearance of logs in a sequence may indicate a performance anomaly corresponding to an abnormal temporal irregularity in a service response. Hence, log anomaly detection is recognized as an efficient mean to perform system anomaly detection [3]. Manually analyzing large and complex log datasets represents a cumbersome and error-prone task [3,8], justifying the need for automated data-driven solutions.…”

Section: Introductionmentioning

confidence: 99%

Application Performance Anomaly Detection with LSTM on Temporal Irregularities in Logs

Baril

Coustié

Mothe

et al. 2020

Proceedings of the 29th ACM International Conference on Information &Amp; Knowledge Management

View full text Add to dashboard Cite

Performance anomalies are a core problem in modern information systems, that affects the execution of the hosted applications. The detection of these anomalies often relies on the analysis of the application execution logs. The current most effective approach is to detect samples that differ from a learnt nominal model. However, current methods often focus on detecting sequential anomalies in logs, neglecting the time elapsed between logs, which is a core component of the performance anomaly detection. In this paper, we develop a new model for performance anomaly detection that captures temporal deviations from the nominal model, by means of a sliding window data representation. This nominal model is trained by a Long Short-Term Memory neural network, which is appropriate to represent complex sequential dependencies. We assess the effectiveness of our model on both simulated and real datasets. We show that it is more robust to temporal variations than current state-of-the-art approaches, while remaining as effective. CCS CONCEPTS• Software and its engineering → Software testing and debugging; • Theory of computation → Semi-supervised learning;• Computing methodologies → Anomaly detection; Neural networks.

show abstract

Online Anomaly Detection in HPC Systems

Cited by 35 publications

References 19 publications

pAElla: Edge AI-Based Real-Time Malware Detection in Data Centers

pAElla: Edge AI-Based Real-Time Malware Detection in Data Centers

Security of HPC Systems: From a Log-analyzing Perspective

Application Performance Anomaly Detection with LSTM on Temporal Irregularities in Logs

Contact Info

Product

Resources

About